[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:09:25 dist-test-slave-3kk6 krb5kdc[16275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:09:25 dist-test-slave-3kk6 krb5kdc[16275](info): set up 2 sockets
Aug 07 15:09:25 dist-test-slave-3kk6 krb5kdc[16275](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 15:09:26.454973 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.000s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:09:26 dist-test-slave-3kk6 krb5kdc[16275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:09:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:09:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:09:26.607445 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:42115
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:42115
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:26.732322 16292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:26.732651 16292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:26.732695 16292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:26.736934 16292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:26.737002 16292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:26.737023 16292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:26.737040 16292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:26.737056 16292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:09:26.742791 16292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:42115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:42115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:26.744254 16292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:26.745564 16292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:26.758594 16299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:26.759490 16298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:26.759647 16292 server_base.cc:1034] running on GCE node
W20260807 15:09:26.759861 16301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:26.760653 16292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:26.762571 16292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:26.763782 16292 hybrid_clock.cc:648] HybridClock initialized: now 1786115366763750 us; error 57 us; skew 500 ppm
Aug 07 15:09:26 dist-test-slave-3kk6 krb5kdc[16275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115366, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:26.767673 16292 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:09:26.769068 16292 webserver.cc:460] Webserver started at http://127.15.226.126:42719/ using document root <none> and password file <none>
I20260807 15:09:26.770041 16292 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:26.770152 16292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:26.770601 16292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:26.773595 16292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "e05cc7e898f94831880dfb7b83267312"
format_stamp: "Formatted at 2026-08-07 15:09:26 on dist-test-slave-3kk6"
server_key: "64729c19fcda56258552fc75dd24d8aa"
server_key_iv: "b6f900c8d9f8f1012b124571b48eadc8"
server_key_version: "encryptionkey@0"
I20260807 15:09:26.774297 16292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "e05cc7e898f94831880dfb7b83267312"
format_stamp: "Formatted at 2026-08-07 15:09:26 on dist-test-slave-3kk6"
server_key: "64729c19fcda56258552fc75dd24d8aa"
server_key_iv: "b6f900c8d9f8f1012b124571b48eadc8"
server_key_version: "encryptionkey@0"
I20260807 15:09:26.779951 16292 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 15:09:26.783948 16310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:26.785638 16292 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:09:26.785848 16292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "e05cc7e898f94831880dfb7b83267312"
format_stamp: "Formatted at 2026-08-07 15:09:26 on dist-test-slave-3kk6"
server_key: "64729c19fcda56258552fc75dd24d8aa"
server_key_iv: "b6f900c8d9f8f1012b124571b48eadc8"
server_key_version: "encryptionkey@0"
I20260807 15:09:26.786005 16292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:26.809433 16292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:26.813932 16292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:26.814316 16292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:26.829768 16292 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:42115
I20260807 15:09:26.829879 16391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:42115 every 8 connection(s)
I20260807 15:09:26.831398 16292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:09:26.836936 16392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:26.837328 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 16292
I20260807 15:09:26.837507 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:09:26.839219 16265 external_mini_cluster.cc:1428] Setting key 4e58b633d6f07c0faf78d65ff70ef280
I20260807 15:09:26.848388 16392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e05cc7e898f94831880dfb7b83267312: Bootstrap starting.
I20260807 15:09:26.852881 16392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e05cc7e898f94831880dfb7b83267312: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:26.854458 16392 log.cc:824] T 00000000000000000000000000000000 P e05cc7e898f94831880dfb7b83267312: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:26.857285 16392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e05cc7e898f94831880dfb7b83267312: No bootstrap required, opened a new log
Aug 07 15:09:26 dist-test-slave-3kk6 krb5kdc[16275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:26.863912 16392 raft_consensus.cc:348] T 00000000000000000000000000000000 P e05cc7e898f94831880dfb7b83267312 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05cc7e898f94831880dfb7b83267312" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 42115 } }
I20260807 15:09:26.864377 16392 raft_consensus.cc:374] T 00000000000000000000000000000000 P e05cc7e898f94831880dfb7b83267312 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:26.864493 16392 raft_consensus.cc:729] T 00000000000000000000000000000000 P e05cc7e898f94831880dfb7b83267312 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e05cc7e898f94831880dfb7b83267312, State: Initialized, Role: FOLLOWER
I20260807 15:09:26.865525 16392 consensus_queue.cc:260] T 00000000000000000000000000000000 P e05cc7e898f94831880dfb7b83267312 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05cc7e898f94831880dfb7b83267312" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 42115 } }
I20260807 15:09:26.865725 16392 raft_consensus.cc:388] T 00000000000000000000000000000000 P e05cc7e898f94831880dfb7b83267312 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:09:26.865906 16392 raft_consensus.cc:482] T 00000000000000000000000000000000 P e05cc7e898f94831880dfb7b83267312 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:09:26.866032 16392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e05cc7e898f94831880dfb7b83267312 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:26.867372 16392 raft_consensus.cc:504] T 00000000000000000000000000000000 P e05cc7e898f94831880dfb7b83267312 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05cc7e898f94831880dfb7b83267312" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 42115 } }
I20260807 15:09:26.868166 16392 leader_election.cc:302] T 00000000000000000000000000000000 P e05cc7e898f94831880dfb7b83267312 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e05cc7e898f94831880dfb7b83267312; no voters:
I20260807 15:09:26.868927 16392 leader_election.cc:288] T 00000000000000000000000000000000 P e05cc7e898f94831880dfb7b83267312 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:09:26.869179 16397 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e05cc7e898f94831880dfb7b83267312 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:26.869583 16397 raft_consensus.cc:686] T 00000000000000000000000000000000 P e05cc7e898f94831880dfb7b83267312 [term 1 LEADER]: Becoming Leader. State: Replica: e05cc7e898f94831880dfb7b83267312, State: Running, Role: LEADER
I20260807 15:09:26.870076 16397 consensus_queue.cc:237] T 00000000000000000000000000000000 P e05cc7e898f94831880dfb7b83267312 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05cc7e898f94831880dfb7b83267312" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 42115 } }
I20260807 15:09:26.870993 16392 sys_catalog.cc:564] T 00000000000000000000000000000000 P e05cc7e898f94831880dfb7b83267312 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:26.872787 16394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:26.844481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53622 (local address 127.15.226.126:42115)
0807 15:09:26.845133 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:26.845152 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:26.846501 (+ 1349us) server_negotiation.cc:407] Connection header received
0807 15:09:26.851659 (+ 5158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:26.851680 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:26.852129 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:26.852536 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:26.855612 (+ 3076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:26.856850 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:26.858286 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:26.858861 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:26.859064 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:26.864545 (+ 5481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:26.864605 (+ 60us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:26.864627 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:26.864675 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:26.867311 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:26.868275 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:26.868281 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:26.868291 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:26.868509 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:26.868973 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:26.868976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:26.868978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:26.869595 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:26.870875 (+ 1280us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:26.871230 (+ 355us) server_negotiation.cc:299] Negotiation successful
0807 15:09:26.871581 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":251,"threads_started":1}
I20260807 15:09:26.872804 16397 sys_catalog.cc:455] T 00000000000000000000000000000000 P e05cc7e898f94831880dfb7b83267312 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e05cc7e898f94831880dfb7b83267312. Latest consensus state: current_term: 1 leader_uuid: "e05cc7e898f94831880dfb7b83267312" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05cc7e898f94831880dfb7b83267312" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 42115 } } }
I20260807 15:09:26.873479 16397 sys_catalog.cc:458] T 00000000000000000000000000000000 P e05cc7e898f94831880dfb7b83267312 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:26.874076 16400 sys_catalog.cc:455] T 00000000000000000000000000000000 P e05cc7e898f94831880dfb7b83267312 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e05cc7e898f94831880dfb7b83267312" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05cc7e898f94831880dfb7b83267312" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 42115 } } }
I20260807 15:09:26.874241 16400 sys_catalog.cc:458] T 00000000000000000000000000000000 P e05cc7e898f94831880dfb7b83267312 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:26.874310 16412 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:09:26.880530 16412 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:09:26.889788 16412 catalog_manager.cc:1347] Generated new cluster ID: 6dc6ee142ec041b59012b577cd8b9668
I20260807 15:09:26.889896 16412 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:09:26.904871 16412 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:09:26.906014 16412 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:09:26.918426 16412 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e05cc7e898f94831880dfb7b83267312: Generated new TSK 0
I20260807 15:09:26.919576 16412 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.226.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
I20260807 15:09:26.966944 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:42115
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:27.139289 16426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:27.139606 16426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:27.139657 16426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:27.143931 16426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:27.144018 16426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:27.144124 16426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:09:27.150276 16426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:42115
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:27.151883 16426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:27.153126 16426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:27.165382 16432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:27.165453 16426 server_base.cc:1034] running on GCE node
W20260807 15:09:27.165342 16431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:27.165431 16435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:27.166095 16426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:27.166790 16426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:27.167968 16426 hybrid_clock.cc:648] HybridClock initialized: now 1786115367167952 us; error 31 us; skew 500 ppm
Aug 07 15:09:27 dist-test-slave-3kk6 krb5kdc[16275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115367, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:27.176302 16426 init.cc:377] Logged in from keytab as kudu/127.15.226.65@KRBTEST.COM (short username kudu)
I20260807 15:09:27.177381 16426 webserver.cc:460] Webserver started at http://127.15.226.65:35237/ using document root <none> and password file <none>
I20260807 15:09:27.178006 16426 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:27.178073 16426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:27.178323 16426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:27.180881 16426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "cff412854c0b481ca3394a1e194eec74"
format_stamp: "Formatted at 2026-08-07 15:09:27 on dist-test-slave-3kk6"
server_key: "477cf3ae5aea6d7c9f1d4f8cf4c9587e"
server_key_iv: "73be81b6d21deb4dbc6fa3df262161a9"
server_key_version: "encryptionkey@0"
I20260807 15:09:27.181593 16426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "cff412854c0b481ca3394a1e194eec74"
format_stamp: "Formatted at 2026-08-07 15:09:27 on dist-test-slave-3kk6"
server_key: "477cf3ae5aea6d7c9f1d4f8cf4c9587e"
server_key_iv: "73be81b6d21deb4dbc6fa3df262161a9"
server_key_version: "encryptionkey@0"
I20260807 15:09:27.185818 16426 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:09:27.188665 16444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:27.190109 16426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:09:27.190248 16426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "cff412854c0b481ca3394a1e194eec74"
format_stamp: "Formatted at 2026-08-07 15:09:27 on dist-test-slave-3kk6"
server_key: "477cf3ae5aea6d7c9f1d4f8cf4c9587e"
server_key_iv: "73be81b6d21deb4dbc6fa3df262161a9"
server_key_version: "encryptionkey@0"
I20260807 15:09:27.190375 16426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:27.215749 16426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:27.219981 16426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:27.220278 16426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:27.221259 16426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:27.222576 16426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:27.222630 16426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:27.222739 16426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:27.222795 16426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:27.243633 16426 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:46393
I20260807 15:09:27.243741 16593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:46393 every 8 connection(s)
I20260807 15:09:27.244793 16426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:09:27.247459 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 16426
I20260807 15:09:27.247584 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:09:27.247921 16265 external_mini_cluster.cc:1428] Setting key 6d56d98470c04756b53765a6dee37254
Aug 07 15:09:27 dist-test-slave-3kk6 krb5kdc[16275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115367, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.226.66@KRBTEST.COM; defaulting to no policy
I20260807 15:09:27.262377 16394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:27.247570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:35821 (local address 127.15.226.126:42115)
0807 15:09:27.247821 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:27.247826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:27.249375 (+ 1549us) server_negotiation.cc:407] Connection header received
0807 15:09:27.250689 (+ 1314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:27.250692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:27.250750 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:27.250837 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:27.252422 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:27.252970 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:27.254357 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:27.254707 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:27.254837 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:27.258287 (+ 3450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:27.258291 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:27.258293 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:27.258319 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:27.260009 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:27.260584 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:27.260587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:27.260588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:27.260669 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:27.261126 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:27.261129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:27.261131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:27.261343 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:27.261452 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:27.262091 (+ 639us) server_negotiation.cc:299] Negotiation successful
0807 15:09:27.262231 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260807 15:09:27.263451 16597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:27.248081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:42115 (local address 127.15.226.65:35821)
0807 15:09:27.249206 (+ 1125us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:27.249273 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:27.250414 (+ 1141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:27.251038 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:27.251046 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:27.251551 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:27.252250 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:27.252268 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:27.253110 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:27.253115 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:27.254214 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:27.254225 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:27.254889 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:27.254893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:27.255133 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:27.255957 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:27.255975 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:27.258082 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:27.260146 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:27.260153 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:27.260170 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:27.260452 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:27.260776 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:27.260840 (+ 64us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:27.260843 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:27.261019 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:27.261467 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:27.261475 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:27.261751 (+ 276us) client_negotiation.cc:769] Sending connection context
0807 15:09:27.262029 (+ 278us) client_negotiation.cc:240] Negotiation successful
0807 15:09:27.262346 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"thread_start_us":262,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.66@KRBTEST.COM" created.
I20260807 15:09:27.265065 16594 heartbeater.cc:347] Connected to a master server at 127.15.226.126:42115
I20260807 15:09:27.265357 16594 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:27.265926 16594 heartbeater.cc:511] Master 127.15.226.126:42115 requested a full tablet report, sending...
I20260807 15:09:27.268344 16334 ts_manager.cc:194] Registered new tserver with Master: cff412854c0b481ca3394a1e194eec74 (127.15.226.65:46393)
I20260807 15:09:27.269830 16334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.65@KRBTEST.COM'} at 127.15.226.65:35821
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
I20260807 15:09:27.286418 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:42115
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:27.436897 16601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:27.437235 16601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:27.437330 16601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:27.442087 16601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:27.442195 16601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:27.442335 16601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:09:27.448704 16601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:42115
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:27.450023 16601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:27.451239 16601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:27.463078 16613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:27.463116 16611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:27.463181 16601 server_base.cc:1034] running on GCE node
W20260807 15:09:27.463083 16609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:27.464020 16601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:27.464819 16601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:27.466037 16601 hybrid_clock.cc:648] HybridClock initialized: now 1786115367466023 us; error 41 us; skew 500 ppm
Aug 07 15:09:27 dist-test-slave-3kk6 krb5kdc[16275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115367, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:27.469468 16601 init.cc:377] Logged in from keytab as kudu/127.15.226.66@KRBTEST.COM (short username kudu)
I20260807 15:09:27.470538 16601 webserver.cc:460] Webserver started at http://127.15.226.66:45705/ using document root <none> and password file <none>
I20260807 15:09:27.471256 16601 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:27.471330 16601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:27.471524 16601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:27.473985 16601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "368d261afbef4d2cacc479ac400d53c1"
format_stamp: "Formatted at 2026-08-07 15:09:27 on dist-test-slave-3kk6"
server_key: "bb4ac6329dcb32273c0e7932117c1af3"
server_key_iv: "1d8b37c236140dc7a9b3cd30c9581304"
server_key_version: "encryptionkey@0"
I20260807 15:09:27.474598 16601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "368d261afbef4d2cacc479ac400d53c1"
format_stamp: "Formatted at 2026-08-07 15:09:27 on dist-test-slave-3kk6"
server_key: "bb4ac6329dcb32273c0e7932117c1af3"
server_key_iv: "1d8b37c236140dc7a9b3cd30c9581304"
server_key_version: "encryptionkey@0"
I20260807 15:09:27.479393 16601 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.002s
I20260807 15:09:27.482237 16623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:27.483707 16601 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:09:27.483862 16601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "368d261afbef4d2cacc479ac400d53c1"
format_stamp: "Formatted at 2026-08-07 15:09:27 on dist-test-slave-3kk6"
server_key: "bb4ac6329dcb32273c0e7932117c1af3"
server_key_iv: "1d8b37c236140dc7a9b3cd30c9581304"
server_key_version: "encryptionkey@0"
I20260807 15:09:27.483976 16601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:27.500361 16601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:27.504050 16601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:27.504222 16601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:27.504802 16601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:27.505697 16601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:27.505759 16601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:27.505812 16601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:27.505829 16601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:27.519317 16601 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:38917
I20260807 15:09:27.519340 16789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:38917 every 8 connection(s)
I20260807 15:09:27.520388 16601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:09:27.526247 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 16601
I20260807 15:09:27.526347 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:09:27.526767 16265 external_mini_cluster.cc:1428] Setting key 9160ec18b7e1180d162453183b5630d9
Aug 07 15:09:27 dist-test-slave-3kk6 krb5kdc[16275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115367, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:27.537621 16394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:27.522767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:33795 (local address 127.15.226.126:42115)
0807 15:09:27.522936 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:27.522941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:27.524090 (+ 1149us) server_negotiation.cc:407] Connection header received
0807 15:09:27.525063 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:27.525066 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:27.525128 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:27.525245 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:27.526804 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:27.527378 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:27.528938 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:27.529167 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:27.529282 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:27.532994 (+ 3712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:27.532997 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:27.532999 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:27.533025 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:27.534772 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:27.535521 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:27.535524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:27.535525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:27.535608 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:27.536090 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:27.536092 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:27.536094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:27.536389 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:27.536475 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:27.537365 (+ 890us) server_negotiation.cc:299] Negotiation successful
0807 15:09:27.537489 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:09:27.539269 16795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:27.523215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:42115 (local address 127.15.226.66:33795)
0807 15:09:27.523919 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:27.523963 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:27.524839 (+ 876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:27.525442 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:27.525450 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:27.525892 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:27.526629 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:27.526664 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:27.527520 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:27.527525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:27.528800 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:27.528810 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:27.529339 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:27.529343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:27.529583 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:27.530379 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:27.530408 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:27.532619 (+ 2211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:27.534929 (+ 2310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:27.534939 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:27.534960 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:27.535402 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:27.535705 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:27.535711 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:27.535715 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:27.535978 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:27.536515 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:27.536526 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:27.536990 (+ 464us) client_negotiation.cc:769] Sending connection context
0807 15:09:27.537306 (+ 316us) client_negotiation.cc:240] Negotiation successful
0807 15:09:27.537825 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":126,"threads_started":1}
WARNING: no policy specified for kudu/127.15.226.67@KRBTEST.COM; defaulting to no policy
I20260807 15:09:27.541019 16790 heartbeater.cc:347] Connected to a master server at 127.15.226.126:42115
I20260807 15:09:27.541294 16790 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:27.541886 16790 heartbeater.cc:511] Master 127.15.226.126:42115 requested a full tablet report, sending...
I20260807 15:09:27.543218 16334 ts_manager.cc:194] Registered new tserver with Master: 368d261afbef4d2cacc479ac400d53c1 (127.15.226.66:38917)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.67@KRBTEST.COM" created.
I20260807 15:09:27.544052 16334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.66@KRBTEST.COM'} at 127.15.226.66:33795
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
I20260807 15:09:27.564003 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:42115
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:27.717731 16800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:27.718112 16800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:27.718209 16800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:27.722496 16800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:27.722635 16800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:27.722774 16800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:09:27.728617 16800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:42115
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:27.729926 16800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:27.731072 16800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:27.742413 16808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:27.742548 16810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:27.742350 16812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:27.742733 16800 server_base.cc:1034] running on GCE node
I20260807 15:09:27.743230 16800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:27.743968 16800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:27.745189 16800 hybrid_clock.cc:648] HybridClock initialized: now 1786115367745158 us; error 45 us; skew 500 ppm
Aug 07 15:09:27 dist-test-slave-3kk6 krb5kdc[16275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115367, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:27.749053 16800 init.cc:377] Logged in from keytab as kudu/127.15.226.67@KRBTEST.COM (short username kudu)
I20260807 15:09:27.750028 16800 webserver.cc:460] Webserver started at http://127.15.226.67:44047/ using document root <none> and password file <none>
I20260807 15:09:27.750645 16800 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:27.750788 16800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:27.751011 16800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:27.753563 16800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "8001981e30754426bc314e19307208ac"
format_stamp: "Formatted at 2026-08-07 15:09:27 on dist-test-slave-3kk6"
server_key: "5eedad81c5bcf0971da89ee791bedcd2"
server_key_iv: "08bd881385ba869c53310259e03d854f"
server_key_version: "encryptionkey@0"
I20260807 15:09:27.754186 16800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "8001981e30754426bc314e19307208ac"
format_stamp: "Formatted at 2026-08-07 15:09:27 on dist-test-slave-3kk6"
server_key: "5eedad81c5bcf0971da89ee791bedcd2"
server_key_iv: "08bd881385ba869c53310259e03d854f"
server_key_version: "encryptionkey@0"
I20260807 15:09:27.758934 16800 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.003s
I20260807 15:09:27.761852 16823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:27.763298 16800 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:09:27.763449 16800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "8001981e30754426bc314e19307208ac"
format_stamp: "Formatted at 2026-08-07 15:09:27 on dist-test-slave-3kk6"
server_key: "5eedad81c5bcf0971da89ee791bedcd2"
server_key_iv: "08bd881385ba869c53310259e03d854f"
server_key_version: "encryptionkey@0"
I20260807 15:09:27.763587 16800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:27.783346 16800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:27.787328 16800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:27.787627 16800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:27.788583 16800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:27.789621 16800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:27.789703 16800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:27.789781 16800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:27.789842 16800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:27.809168 16800 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:39685
I20260807 15:09:27.809228 16982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:39685 every 8 connection(s)
I20260807 15:09:27.810269 16800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:09:27.813498 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 16800
I20260807 15:09:27.813683 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:09:27.814077 16265 external_mini_cluster.cc:1428] Setting key 74c787abef96dabd3782b4cdbb94f6f8
Aug 07 15:09:27 dist-test-slave-3kk6 krb5kdc[16275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115367, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:27.827585 16394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:27.812466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:40053 (local address 127.15.226.126:42115)
0807 15:09:27.812630 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:27.812634 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:27.814590 (+ 1956us) server_negotiation.cc:407] Connection header received
0807 15:09:27.816157 (+ 1567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:27.816160 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:27.816225 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:27.816314 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:27.817966 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:27.818586 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:27.820235 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:27.820506 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:27.820634 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:27.823662 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:27.823665 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:27.823668 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:27.823698 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:27.825131 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:27.825746 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:27.825750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:27.825752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:27.825849 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:27.826254 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:27.826259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:27.826262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:27.826492 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:27.826617 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:27.827240 (+ 623us) server_negotiation.cc:299] Negotiation successful
0807 15:09:27.827398 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:09:27.828774 16988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:27.812855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:42115 (local address 127.15.226.67:40053)
0807 15:09:27.814370 (+ 1515us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:27.814425 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:27.815903 (+ 1478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:27.816517 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:27.816526 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:27.817119 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:27.817776 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:27.817793 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:27.818774 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:27.818783 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:27.820077 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:27.820090 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:27.820701 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:27.820706 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:27.820936 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:27.821610 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:27.821629 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:27.823438 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:27.825283 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:27.825290 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:27.825310 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:27.825624 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:27.825963 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:27.825968 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:27.825970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:27.826141 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:27.826615 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:27.826625 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:27.826967 (+ 342us) client_negotiation.cc:769] Sending connection context
0807 15:09:27.827194 (+ 227us) client_negotiation.cc:240] Negotiation successful
0807 15:09:27.827585 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1215,"thread_start_us":986,"threads_started":1}
I20260807 15:09:27.830215 16983 heartbeater.cc:347] Connected to a master server at 127.15.226.126:42115
I20260807 15:09:27.830518 16983 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:27.831178 16983 heartbeater.cc:511] Master 127.15.226.126:42115 requested a full tablet report, sending...
I20260807 15:09:27.832429 16334 ts_manager.cc:194] Registered new tserver with Master: 8001981e30754426bc314e19307208ac (127.15.226.67:39685)
I20260807 15:09:27.833040 16334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.67@KRBTEST.COM'} at 127.15.226.67:40053
I20260807 15:09:27.841795 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 15:09:27 dist-test-slave-3kk6 krb5kdc[16275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115367, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 15:09:27 dist-test-slave-3kk6 krb5kdc[16275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115367, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.226.65@KRBTEST.COM
I20260807 15:09:27.872575 16994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:27.862533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46956 (local address 127.15.226.65:46393)
0807 15:09:27.862954 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:27.862960 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:27.862988 (+ 28us) server_negotiation.cc:407] Connection header received
0807 15:09:27.863146 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:27.863152 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:27.863350 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:27.863492 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:27.864241 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:27.865043 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:27.865918 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:27.866289 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:27.866371 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:27.869073 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:27.869083 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:27.869097 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:27.869124 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:27.870767 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:27.871229 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:27.871235 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:27.871236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:27.871327 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:27.871564 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:27.871566 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:27.871567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:27.871826 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:27.872023 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:27.872274 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 15:09:27.872420 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":223,"threads_started":1}
W20260807 15:09:27.873792 16480 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46956
Aug 07 15:09:27 dist-test-slave-3kk6 krb5kdc[16275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 15:09:27 dist-test-slave-3kk6 krb5kdc[16275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.65@KRBTEST.COM
I20260807 15:09:27.904325 16994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:27.895253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46968 (local address 127.15.226.65:46393)
0807 15:09:27.895417 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:27.895425 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:27.895592 (+ 167us) server_negotiation.cc:407] Connection header received
0807 15:09:27.895713 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:27.895716 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:27.895770 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:27.895886 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:27.896588 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:27.897163 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:27.898126 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:27.898379 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:27.898471 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:27.901180 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:27.901183 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:27.901185 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:27.901214 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:27.902635 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:27.903140 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:27.903142 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:27.903144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:27.903229 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:27.903470 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:27.903472 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:27.903473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:27.903657 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:27.903770 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:27.903977 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 15:09:27.904095 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:09:27.905629 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 16426
I20260807 15:09:27.919539 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 16601
I20260807 15:09:27.936900 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 16800
I20260807 15:09:27.943331 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 16292
2026-08-07T15:09:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2800 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:09:28 dist-test-slave-3kk6 krb5kdc[17005](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:09:28 dist-test-slave-3kk6 krb5kdc[17005](info): set up 2 sockets
Aug 07 15:09:28 dist-test-slave-3kk6 krb5kdc[17005](info): commencing operation
krb5kdc: starting...
W20260807 15:09:29.179147 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:09:29 dist-test-slave-3kk6 krb5kdc[17005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:09:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:09:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:09:29.318517 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:46823
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:33485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:46823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:29.442349 17022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:29.442746 17022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:29.442844 17022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:29.447110 17022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:29.447237 17022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:29.447324 17022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:29.447386 17022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:29.447429 17022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:09:29.453183 17022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:33485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:46823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:46823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:29.454493 17022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:29.455662 17022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:29.464711 17027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:29.464730 17031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:29.464779 17029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:29.465030 17022 server_base.cc:1034] running on GCE node
I20260807 15:09:29.465773 17022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:29.467057 17022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:29.468222 17022 hybrid_clock.cc:648] HybridClock initialized: now 1786115369468208 us; error 30 us; skew 500 ppm
Aug 07 15:09:29 dist-test-slave-3kk6 krb5kdc[17005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115369, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:29.471210 17022 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:09:29.472131 17022 webserver.cc:460] Webserver started at http://127.15.226.126:45681/ using document root <none> and password file <none>
I20260807 15:09:29.472719 17022 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:29.472815 17022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:29.473065 17022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:29.475266 17022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "f69f72ae73834b1d83aabd79b68ef115"
format_stamp: "Formatted at 2026-08-07 15:09:29 on dist-test-slave-3kk6"
server_key: "8d4a5f67df0653df0d08f6c75ca76c58"
server_key_iv: "83a01b43af9f9110b968ae7bc3ce61bc"
server_key_version: "encryptionkey@0"
I20260807 15:09:29.475947 17022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "f69f72ae73834b1d83aabd79b68ef115"
format_stamp: "Formatted at 2026-08-07 15:09:29 on dist-test-slave-3kk6"
server_key: "8d4a5f67df0653df0d08f6c75ca76c58"
server_key_iv: "83a01b43af9f9110b968ae7bc3ce61bc"
server_key_version: "encryptionkey@0"
I20260807 15:09:29.479969 17022 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:09:29.482517 17044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:29.483822 17022 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:09:29.483999 17022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "f69f72ae73834b1d83aabd79b68ef115"
format_stamp: "Formatted at 2026-08-07 15:09:29 on dist-test-slave-3kk6"
server_key: "8d4a5f67df0653df0d08f6c75ca76c58"
server_key_iv: "83a01b43af9f9110b968ae7bc3ce61bc"
server_key_version: "encryptionkey@0"
I20260807 15:09:29.484128 17022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:29.507257 17022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:29.511139 17022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:29.511389 17022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:29.519027 17022 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:46823
I20260807 15:09:29.519058 17122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:46823 every 8 connection(s)
I20260807 15:09:29.520074 17022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:09:29.523113 17125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:29.527287 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 17022
I20260807 15:09:29.527443 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:09:29.527705 16265 external_mini_cluster.cc:1428] Setting key a760754df52c79f52722dced768d4672
I20260807 15:09:29.529417 17125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f69f72ae73834b1d83aabd79b68ef115: Bootstrap starting.
I20260807 15:09:29.531903 17125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f69f72ae73834b1d83aabd79b68ef115: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:29.532747 17125 log.cc:824] T 00000000000000000000000000000000 P f69f72ae73834b1d83aabd79b68ef115: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:29.535161 17125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f69f72ae73834b1d83aabd79b68ef115: No bootstrap required, opened a new log
Aug 07 15:09:29 dist-test-slave-3kk6 krb5kdc[17005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:29.539899 17125 raft_consensus.cc:348] T 00000000000000000000000000000000 P f69f72ae73834b1d83aabd79b68ef115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f69f72ae73834b1d83aabd79b68ef115" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46823 } }
I20260807 15:09:29.540148 17125 raft_consensus.cc:374] T 00000000000000000000000000000000 P f69f72ae73834b1d83aabd79b68ef115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:29.540236 17125 raft_consensus.cc:729] T 00000000000000000000000000000000 P f69f72ae73834b1d83aabd79b68ef115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f69f72ae73834b1d83aabd79b68ef115, State: Initialized, Role: FOLLOWER
I20260807 15:09:29.540733 17125 consensus_queue.cc:260] T 00000000000000000000000000000000 P f69f72ae73834b1d83aabd79b68ef115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f69f72ae73834b1d83aabd79b68ef115" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46823 } }
I20260807 15:09:29.540871 17125 raft_consensus.cc:388] T 00000000000000000000000000000000 P f69f72ae73834b1d83aabd79b68ef115 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:09:29.540962 17125 raft_consensus.cc:482] T 00000000000000000000000000000000 P f69f72ae73834b1d83aabd79b68ef115 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:09:29.541066 17125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f69f72ae73834b1d83aabd79b68ef115 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:29.542215 17125 raft_consensus.cc:504] T 00000000000000000000000000000000 P f69f72ae73834b1d83aabd79b68ef115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f69f72ae73834b1d83aabd79b68ef115" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46823 } }
I20260807 15:09:29.542593 17125 leader_election.cc:302] T 00000000000000000000000000000000 P f69f72ae73834b1d83aabd79b68ef115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f69f72ae73834b1d83aabd79b68ef115; no voters:
I20260807 15:09:29.542994 17125 leader_election.cc:288] T 00000000000000000000000000000000 P f69f72ae73834b1d83aabd79b68ef115 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:09:29.543113 17134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f69f72ae73834b1d83aabd79b68ef115 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:29.543320 17134 raft_consensus.cc:686] T 00000000000000000000000000000000 P f69f72ae73834b1d83aabd79b68ef115 [term 1 LEADER]: Becoming Leader. State: Replica: f69f72ae73834b1d83aabd79b68ef115, State: Running, Role: LEADER
I20260807 15:09:29.543574 17132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:29.529193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45734 (local address 127.15.226.126:46823)
0807 15:09:29.529743 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:29.529760 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:29.529799 (+ 39us) server_negotiation.cc:407] Connection header received
0807 15:09:29.530474 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:29.530492 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:29.530904 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:29.531252 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:29.532106 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:29.533210 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:29.534459 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:29.534992 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:29.535129 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:29.538084 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:29.538089 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:29.538104 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:29.538133 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:29.539952 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:29.540496 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:29.540501 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:29.540506 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:29.540621 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:29.540976 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:29.540979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:29.540981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:29.541476 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:29.541701 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:29.542127 (+ 426us) server_negotiation.cc:299] Negotiation successful
0807 15:09:29.542497 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":158,"threads_started":1}
I20260807 15:09:29.543875 17134 consensus_queue.cc:237] T 00000000000000000000000000000000 P f69f72ae73834b1d83aabd79b68ef115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f69f72ae73834b1d83aabd79b68ef115" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46823 } }
I20260807 15:09:29.544396 17125 sys_catalog.cc:564] T 00000000000000000000000000000000 P f69f72ae73834b1d83aabd79b68ef115 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:29.545586 17137 sys_catalog.cc:455] T 00000000000000000000000000000000 P f69f72ae73834b1d83aabd79b68ef115 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f69f72ae73834b1d83aabd79b68ef115. Latest consensus state: current_term: 1 leader_uuid: "f69f72ae73834b1d83aabd79b68ef115" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f69f72ae73834b1d83aabd79b68ef115" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46823 } } }
I20260807 15:09:29.545794 17137 sys_catalog.cc:458] T 00000000000000000000000000000000 P f69f72ae73834b1d83aabd79b68ef115 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:29.546195 17136 sys_catalog.cc:455] T 00000000000000000000000000000000 P f69f72ae73834b1d83aabd79b68ef115 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f69f72ae73834b1d83aabd79b68ef115" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f69f72ae73834b1d83aabd79b68ef115" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46823 } } }
I20260807 15:09:29.546321 17146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:09:29.546350 17136 sys_catalog.cc:458] T 00000000000000000000000000000000 P f69f72ae73834b1d83aabd79b68ef115 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:29.549674 17146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:09:29.555397 17146 catalog_manager.cc:1347] Generated new cluster ID: bbf3415e546e437198ea97acb8b2eb73
I20260807 15:09:29.555495 17146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:09:29.567050 17146 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:09:29.568110 17146 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:09:29.579049 17146 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f69f72ae73834b1d83aabd79b68ef115: Generated new TSK 0
I20260807 15:09:29.579975 17146 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.226.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
I20260807 15:09:29.618584 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:46823
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:33485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:29.740013 17162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:29.740311 17162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:29.740360 17162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:29.744663 17162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:29.744748 17162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:29.744814 17162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:09:29.750444 17162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:33485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:46823
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:29.751664 17162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:29.752719 17162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:09:29.762826 17162 server_base.cc:1034] running on GCE node
W20260807 15:09:29.762773 17174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:29.763558 17170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:29.763589 17167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:29.764053 17162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:29.764679 17162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:29.765873 17162 hybrid_clock.cc:648] HybridClock initialized: now 1786115369765864 us; error 33 us; skew 500 ppm
Aug 07 15:09:29 dist-test-slave-3kk6 krb5kdc[17005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115369, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:29.768966 17162 init.cc:377] Logged in from keytab as kudu/127.15.226.65@KRBTEST.COM (short username kudu)
I20260807 15:09:29.769920 17162 webserver.cc:460] Webserver started at http://127.15.226.65:32769/ using document root <none> and password file <none>
I20260807 15:09:29.770525 17162 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:29.770623 17162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:29.770900 17162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:29.773090 17162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "fc7440876978457a9a615de634184d8a"
format_stamp: "Formatted at 2026-08-07 15:09:29 on dist-test-slave-3kk6"
server_key: "b21ab476149325e45612b3c524ef379a"
server_key_iv: "cfed1fac3f8f964b9f01c4d8abacef9a"
server_key_version: "encryptionkey@0"
I20260807 15:09:29.773706 17162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "fc7440876978457a9a615de634184d8a"
format_stamp: "Formatted at 2026-08-07 15:09:29 on dist-test-slave-3kk6"
server_key: "b21ab476149325e45612b3c524ef379a"
server_key_iv: "cfed1fac3f8f964b9f01c4d8abacef9a"
server_key_version: "encryptionkey@0"
I20260807 15:09:29.777683 17162 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:09:29.780254 17185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:29.781524 17162 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:09:29.781687 17162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "fc7440876978457a9a615de634184d8a"
format_stamp: "Formatted at 2026-08-07 15:09:29 on dist-test-slave-3kk6"
server_key: "b21ab476149325e45612b3c524ef379a"
server_key_iv: "cfed1fac3f8f964b9f01c4d8abacef9a"
server_key_version: "encryptionkey@0"
I20260807 15:09:29.781816 17162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:29.805693 17162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:29.809371 17162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:29.809641 17162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:29.810321 17162 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:29.811295 17162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:29.811398 17162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:29.811480 17162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:29.811535 17162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:29.821260 17162 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:41035
I20260807 15:09:29.821298 17343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:41035 every 8 connection(s)
I20260807 15:09:29.822293 17162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:09:29.827001 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 17162
I20260807 15:09:29.827140 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:09:29.827435 16265 external_mini_cluster.cc:1428] Setting key 98309e5c3eb90fce7c3899ef0ec51db0
Aug 07 15:09:29 dist-test-slave-3kk6 krb5kdc[17005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115369, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:29.838994 17132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:29.824391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:59977 (local address 127.15.226.126:46823)
0807 15:09:29.824539 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:29.824543 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:29.825476 (+ 933us) server_negotiation.cc:407] Connection header received
0807 15:09:29.827320 (+ 1844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:29.827324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:29.827395 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:29.827502 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:29.828966 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:29.829739 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:29.831197 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:29.831448 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:29.831576 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:29.834596 (+ 3020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:29.834602 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:29.834608 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:29.834645 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:29.836531 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:29.837100 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:29.837106 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:29.837111 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:29.837234 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:29.837597 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:29.837600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:29.837605 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:29.837881 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:29.838007 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:29.838566 (+ 559us) server_negotiation.cc:299] Negotiation successful
0807 15:09:29.838772 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 15:09:29.839852 17348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:29.824806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46823 (local address 127.15.226.65:59977)
0807 15:09:29.825298 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:29.825346 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:29.827053 (+ 1707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:29.827716 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:29.827723 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:29.828221 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:29.828787 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:29.828807 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:29.829912 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:29.829915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:29.831052 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:29.831064 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:29.831627 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:29.831632 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:29.831825 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:29.832544 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:29.832561 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:29.834391 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:29.836673 (+ 2282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:29.836679 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:29.836693 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:29.836983 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:29.837342 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:29.837344 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:29.837346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:29.837497 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:29.838006 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:29.838012 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:29.838279 (+ 267us) client_negotiation.cc:769] Sending connection context
0807 15:09:29.838464 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 15:09:29.838855 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":128,"threads_started":1}
WARNING: no policy specified for kudu/127.15.226.66@KRBTEST.COM; defaulting to no policy
I20260807 15:09:29.841291 17345 heartbeater.cc:347] Connected to a master server at 127.15.226.126:46823
I20260807 15:09:29.841586 17345 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:29.842175 17345 heartbeater.cc:511] Master 127.15.226.126:46823 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.66@KRBTEST.COM" created.
I20260807 15:09:29.844025 17064 ts_manager.cc:194] Registered new tserver with Master: fc7440876978457a9a615de634184d8a (127.15.226.65:41035)
I20260807 15:09:29.845278 17064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.65@KRBTEST.COM'} at 127.15.226.65:59977
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
I20260807 15:09:29.862174 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:46823
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:33485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:29.983210 17353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:29.983544 17353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:29.983595 17353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:29.988080 17353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:29.988210 17353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:29.988281 17353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:09:29.994078 17353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:33485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:46823
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:29.995293 17353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:29.996361 17353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:09:30.006860 17353 server_base.cc:1034] running on GCE node
W20260807 15:09:30.006793 17360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:30.006789 17359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:30.006783 17362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:30.007606 17353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:30.008353 17353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:30.009557 17353 hybrid_clock.cc:648] HybridClock initialized: now 1786115370009542 us; error 33 us; skew 500 ppm
Aug 07 15:09:30 dist-test-slave-3kk6 krb5kdc[17005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115370, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:30.012576 17353 init.cc:377] Logged in from keytab as kudu/127.15.226.66@KRBTEST.COM (short username kudu)
I20260807 15:09:30.013720 17353 webserver.cc:460] Webserver started at http://127.15.226.66:40759/ using document root <none> and password file <none>
I20260807 15:09:30.014343 17353 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:30.014421 17353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:30.014606 17353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:30.016878 17353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "bf3cf4eb515647349b644f04c68f91e5"
format_stamp: "Formatted at 2026-08-07 15:09:30 on dist-test-slave-3kk6"
server_key: "2b9d5377270009304a464566c274b993"
server_key_iv: "b213a76ff96e63328f31cac540797b97"
server_key_version: "encryptionkey@0"
I20260807 15:09:30.017478 17353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "bf3cf4eb515647349b644f04c68f91e5"
format_stamp: "Formatted at 2026-08-07 15:09:30 on dist-test-slave-3kk6"
server_key: "2b9d5377270009304a464566c274b993"
server_key_iv: "b213a76ff96e63328f31cac540797b97"
server_key_version: "encryptionkey@0"
I20260807 15:09:30.021591 17353 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:09:30.024346 17371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:30.025671 17353 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:09:30.025802 17353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "bf3cf4eb515647349b644f04c68f91e5"
format_stamp: "Formatted at 2026-08-07 15:09:30 on dist-test-slave-3kk6"
server_key: "2b9d5377270009304a464566c274b993"
server_key_iv: "b213a76ff96e63328f31cac540797b97"
server_key_version: "encryptionkey@0"
I20260807 15:09:30.025933 17353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:30.042963 17353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:30.046897 17353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:30.047115 17353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:30.047787 17353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:30.048770 17353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:30.048840 17353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:30.048969 17353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:30.049014 17353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:30.058703 17353 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:37381
I20260807 15:09:30.058732 17529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:37381 every 8 connection(s)
I20260807 15:09:30.059702 17353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:09:30.060449 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 17353
I20260807 15:09:30.060571 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:09:30.060853 16265 external_mini_cluster.cc:1428] Setting key 01b7795d0d2a231a606c6f4ce85e93b9
Aug 07 15:09:30 dist-test-slave-3kk6 krb5kdc[17005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115370, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.226.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.67@KRBTEST.COM" created.
I20260807 15:09:30.077203 17132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:30.061717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:44661 (local address 127.15.226.126:46823)
0807 15:09:30.061919 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:30.061924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:30.062924 (+ 1000us) server_negotiation.cc:407] Connection header received
0807 15:09:30.064262 (+ 1338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:30.064269 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:30.064334 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:30.064442 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:30.066181 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.066822 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.068842 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.069072 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.069152 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:30.072768 (+ 3616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:30.072772 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:30.072774 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:30.072802 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:30.074474 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:30.075086 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:30.075088 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:30.075090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:30.075174 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:30.075645 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:30.075648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:30.075649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:30.075853 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:30.075997 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:30.076756 (+ 759us) server_negotiation.cc:299] Negotiation successful
0807 15:09:30.076944 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260807 15:09:30.078061 17534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:30.062153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46823 (local address 127.15.226.66:44661)
0807 15:09:30.062741 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:30.062798 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:30.064047 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:30.064672 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:30.064684 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:30.065235 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:30.066000 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:30.066025 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.066983 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.066990 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:30.068678 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:30.068696 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.069252 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.069258 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:30.069559 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:30.070286 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:30.070314 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:30.072484 (+ 2170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:30.074605 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:30.074610 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:30.074627 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:30.074986 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:30.075306 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:30.075309 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:30.075310 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:30.075499 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:30.076022 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:30.076031 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:30.076439 (+ 408us) client_negotiation.cc:769] Sending connection context
0807 15:09:30.076645 (+ 206us) client_negotiation.cc:240] Negotiation successful
0807 15:09:30.077015 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":174,"threads_started":1}
I20260807 15:09:30.079664 17530 heartbeater.cc:347] Connected to a master server at 127.15.226.126:46823
I20260807 15:09:30.079958 17530 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:30.080673 17530 heartbeater.cc:511] Master 127.15.226.126:46823 requested a full tablet report, sending...
I20260807 15:09:30.082079 17064 ts_manager.cc:194] Registered new tserver with Master: bf3cf4eb515647349b644f04c68f91e5 (127.15.226.66:37381)
I20260807 15:09:30.083029 17064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.66@KRBTEST.COM'} at 127.15.226.66:44661
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
I20260807 15:09:30.098289 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:46823
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:33485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:30.222198 17537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:30.222577 17537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:30.222694 17537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:30.227785 17537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:30.227891 17537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:30.227999 17537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:09:30.233835 17537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:33485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:46823
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:30.235191 17537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:30.236313 17537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:09:30.247010 17537 server_base.cc:1034] running on GCE node
W20260807 15:09:30.246966 17546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:30.246996 17543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:30.247021 17544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:30.247787 17537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:30.248536 17537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:30.249732 17537 hybrid_clock.cc:648] HybridClock initialized: now 1786115370249712 us; error 37 us; skew 500 ppm
Aug 07 15:09:30 dist-test-slave-3kk6 krb5kdc[17005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115370, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:30.252698 17537 init.cc:377] Logged in from keytab as kudu/127.15.226.67@KRBTEST.COM (short username kudu)
I20260807 15:09:30.253597 17537 webserver.cc:460] Webserver started at http://127.15.226.67:41507/ using document root <none> and password file <none>
I20260807 15:09:30.254204 17537 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:30.254269 17537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:30.254459 17537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:30.256791 17537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "ee36e3be2bee4d98b34305bbde38a7c0"
format_stamp: "Formatted at 2026-08-07 15:09:30 on dist-test-slave-3kk6"
server_key: "8027385026c3ea8228e010b709efd92a"
server_key_iv: "18cd492c7f79d14433dc3d5078d45da3"
server_key_version: "encryptionkey@0"
I20260807 15:09:30.257395 17537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "ee36e3be2bee4d98b34305bbde38a7c0"
format_stamp: "Formatted at 2026-08-07 15:09:30 on dist-test-slave-3kk6"
server_key: "8027385026c3ea8228e010b709efd92a"
server_key_iv: "18cd492c7f79d14433dc3d5078d45da3"
server_key_version: "encryptionkey@0"
I20260807 15:09:30.261529 17537 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:09:30.264385 17554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:30.265651 17537 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:09:30.265779 17537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "ee36e3be2bee4d98b34305bbde38a7c0"
format_stamp: "Formatted at 2026-08-07 15:09:30 on dist-test-slave-3kk6"
server_key: "8027385026c3ea8228e010b709efd92a"
server_key_iv: "18cd492c7f79d14433dc3d5078d45da3"
server_key_version: "encryptionkey@0"
I20260807 15:09:30.265903 17537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:30.276631 17537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:30.279999 17537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:30.280216 17537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:30.280858 17537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:30.281726 17537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:30.281776 17537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:30.281847 17537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:30.281885 17537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:30.292644 17537 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:37661
I20260807 15:09:30.292662 17708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:37661 every 8 connection(s)
I20260807 15:09:30.293658 17537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:09:30.297572 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 17537
I20260807 15:09:30.297739 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:09:30.298032 16265 external_mini_cluster.cc:1428] Setting key aa0d127a0ce9c0a802ca3a9d23c5f300
Aug 07 15:09:30 dist-test-slave-3kk6 krb5kdc[17005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115370, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:30.309266 17132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:30.295785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:53173 (local address 127.15.226.126:46823)
0807 15:09:30.295948 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:30.295952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:30.296867 (+ 915us) server_negotiation.cc:407] Connection header received
0807 15:09:30.298694 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:30.298698 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:30.298769 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:30.298879 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:30.300539 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.301096 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.302346 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.302570 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.302718 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:30.305711 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:30.305715 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:30.305718 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:30.305744 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:30.307157 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:30.307735 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:30.307738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:30.307739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:30.307814 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:30.308160 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:30.308163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:30.308164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:30.308349 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:30.308438 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:30.308962 (+ 524us) server_negotiation.cc:299] Negotiation successful
0807 15:09:30.309135 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 15:09:30.310341 17714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:30.296224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46823 (local address 127.15.226.67:53173)
0807 15:09:30.296700 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:30.296747 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:30.298427 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:30.299141 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:30.299149 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:30.299657 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:30.300384 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:30.300406 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.301216 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.301218 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:30.302217 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:30.302227 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.302779 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.302783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:30.302981 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:30.303736 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:30.303755 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:30.305513 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:30.307312 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:30.307318 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:30.307334 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:30.307633 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:30.307904 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:30.307906 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:30.307908 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:30.308074 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:30.308454 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:30.308460 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:30.308733 (+ 273us) client_negotiation.cc:769] Sending connection context
0807 15:09:30.308937 (+ 204us) client_negotiation.cc:240] Negotiation successful
0807 15:09:30.309280 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":131,"threads_started":1}
I20260807 15:09:30.312000 17710 heartbeater.cc:347] Connected to a master server at 127.15.226.126:46823
I20260807 15:09:30.312312 17710 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:30.312987 17710 heartbeater.cc:511] Master 127.15.226.126:46823 requested a full tablet report, sending...
I20260807 15:09:30.314060 17065 ts_manager.cc:194] Registered new tserver with Master: ee36e3be2bee4d98b34305bbde38a7c0 (127.15.226.67:37661)
I20260807 15:09:30.314777 17065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.67@KRBTEST.COM'} at 127.15.226.67:53173
I20260807 15:09:30.324930 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:09:30.337879 17132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:30.329646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45746 (local address 127.15.226.126:46823)
0807 15:09:30.329795 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:30.329802 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:30.330028 (+ 226us) server_negotiation.cc:407] Connection header received
0807 15:09:30.330208 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:30.330211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:30.330266 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:30.330353 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:30.331244 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.331801 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.332663 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.332896 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.333011 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:30.334527 (+ 1516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:30.334535 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:30.334537 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:30.334563 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:30.336005 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:30.336475 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:30.336478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:30.336480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:30.336574 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:30.336892 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:30.336895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:30.336897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:30.337153 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:30.337306 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:30.337565 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 15:09:30.337691 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 15:09:30.344990 17065 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45746:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:09:30.348718 17065 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:09:30.365656 17733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:30.359579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44750 (local address 127.15.226.65:41035)
0807 15:09:30.360239 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:30.360246 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:30.360847 (+ 601us) server_negotiation.cc:407] Connection header received
0807 15:09:30.360934 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:30.360939 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:30.361135 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:30.361326 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:30.361723 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.362529 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.364303 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.364895 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.365043 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:30.365284 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:30.365395 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 15:09:30.365491 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":101,"threads_started":1}
I20260807 15:09:30.365648 17730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:30.359362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:41035 (local address 127.0.0.1:44750)
0807 15:09:30.360492 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:30.360547 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:30.360696 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:30.361481 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:30.361484 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:30.361499 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:30.361601 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:30.361607 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.362675 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.362681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:30.364166 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:30.364178 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.365042 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.365046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:30.365181 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:30.365223 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 15:09:30.365359 (+ 136us) client_negotiation.cc:240] Negotiation successful
0807 15:09:30.365488 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":924,"mutex_wait_us":32,"spinlock_wait_cycles":8064,"thread_start_us":91,"threads_started":1}
I20260807 15:09:30.366080 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:30.359548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:37381 (local address 127.0.0.1:41906)
0807 15:09:30.360491 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:30.360549 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:30.360697 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:30.361304 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:30.361310 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:30.361334 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:30.361485 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:30.361492 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.362991 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.362995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:30.364166 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:30.364178 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.365540 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.365544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:30.365727 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:30.365772 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 15:09:30.365844 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 15:09:30.365921 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":767,"mutex_wait_us":104,"spinlock_wait_cycles":7040,"thread_start_us":123,"threads_started":1}
I20260807 15:09:30.366333 17739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:30.360216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41906 (local address 127.15.226.66:37381)
0807 15:09:30.360673 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:30.360685 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:30.360709 (+ 24us) server_negotiation.cc:407] Connection header received
0807 15:09:30.360893 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:30.360900 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:30.361130 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:30.361293 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:30.361614 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.362854 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.364337 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.365255 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.365347 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:30.365470 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:30.366096 (+ 626us) server_negotiation.cc:299] Negotiation successful
0807 15:09:30.366195 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":167,"threads_started":1}
I20260807 15:09:30.366431 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:30.359763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:37661 (local address 127.0.0.1:48664)
0807 15:09:30.360894 (+ 1131us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:30.360907 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:30.360987 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:30.361675 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:30.361680 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:30.361700 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:30.361778 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:30.361784 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.363208 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.363212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:30.364206 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:30.364216 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.365540 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.365544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:30.365732 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:30.366015 (+ 283us) client_negotiation.cc:769] Sending connection context
0807 15:09:30.366085 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 15:09:30.366274 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1044,"mutex_wait_us":280,"thread_start_us":129,"threads_started":1}
I20260807 15:09:30.367103 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:30.360216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48664 (local address 127.15.226.67:37661)
0807 15:09:30.361245 (+ 1029us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:30.361257 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:30.361278 (+ 21us) server_negotiation.cc:407] Connection header received
0807 15:09:30.361351 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:30.361357 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:30.361525 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:30.361691 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:30.361902 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.363034 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.364337 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.365188 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.365328 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:30.365470 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:30.366870 (+ 1400us) server_negotiation.cc:299] Negotiation successful
0807 15:09:30.366969 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":909,"thread_start_us":109,"threads_started":1}
I20260807 15:09:30.369395 17250 tablet_service.cc:1467] Processing CreateTablet for tablet 4e4f5e1dbff84fb28bbcec4e9c879d1f (DEFAULT_TABLE table=test-table [id=c1531d38f9ed484dad66e247428e43c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:30.369447 17621 tablet_service.cc:1467] Processing CreateTablet for tablet 4e4f5e1dbff84fb28bbcec4e9c879d1f (DEFAULT_TABLE table=test-table [id=c1531d38f9ed484dad66e247428e43c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:30.370461 17250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e4f5e1dbff84fb28bbcec4e9c879d1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:30.370471 17621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e4f5e1dbff84fb28bbcec4e9c879d1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:30.372599 17436 tablet_service.cc:1467] Processing CreateTablet for tablet 4e4f5e1dbff84fb28bbcec4e9c879d1f (DEFAULT_TABLE table=test-table [id=c1531d38f9ed484dad66e247428e43c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:30.373752 17436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e4f5e1dbff84fb28bbcec4e9c879d1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:30.377542 17740 tablet_bootstrap.cc:492] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P ee36e3be2bee4d98b34305bbde38a7c0: Bootstrap starting.
I20260807 15:09:30.379109 17741 tablet_bootstrap.cc:492] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a: Bootstrap starting.
I20260807 15:09:30.380477 17740 tablet_bootstrap.cc:654] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P ee36e3be2bee4d98b34305bbde38a7c0: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:30.380731 17742 tablet_bootstrap.cc:492] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P bf3cf4eb515647349b644f04c68f91e5: Bootstrap starting.
I20260807 15:09:30.381605 17741 tablet_bootstrap.cc:654] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:30.382967 17741 log.cc:824] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:30.383160 17740 log.cc:824] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P ee36e3be2bee4d98b34305bbde38a7c0: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:30.383721 17742 tablet_bootstrap.cc:654] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P bf3cf4eb515647349b644f04c68f91e5: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:30.384603 17742 log.cc:824] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P bf3cf4eb515647349b644f04c68f91e5: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:30.385331 17740 tablet_bootstrap.cc:492] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P ee36e3be2bee4d98b34305bbde38a7c0: No bootstrap required, opened a new log
I20260807 15:09:30.385331 17741 tablet_bootstrap.cc:492] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a: No bootstrap required, opened a new log
I20260807 15:09:30.385543 17741 ts_tablet_manager.cc:1397] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260807 15:09:30.385543 17740 ts_tablet_manager.cc:1397] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P ee36e3be2bee4d98b34305bbde38a7c0: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20260807 15:09:30.387223 17742 tablet_bootstrap.cc:492] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P bf3cf4eb515647349b644f04c68f91e5: No bootstrap required, opened a new log
I20260807 15:09:30.387426 17742 ts_tablet_manager.cc:1397] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P bf3cf4eb515647349b644f04c68f91e5: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 15:09:30.391182 17741 raft_consensus.cc:348] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc7440876978457a9a615de634184d8a" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 41035 } } peers { permanent_uuid: "bf3cf4eb515647349b644f04c68f91e5" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 37381 } } peers { permanent_uuid: "ee36e3be2bee4d98b34305bbde38a7c0" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37661 } }
I20260807 15:09:30.391182 17740 raft_consensus.cc:348] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P ee36e3be2bee4d98b34305bbde38a7c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc7440876978457a9a615de634184d8a" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 41035 } } peers { permanent_uuid: "bf3cf4eb515647349b644f04c68f91e5" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 37381 } } peers { permanent_uuid: "ee36e3be2bee4d98b34305bbde38a7c0" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37661 } }
I20260807 15:09:30.391520 17740 raft_consensus.cc:374] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P ee36e3be2bee4d98b34305bbde38a7c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:30.391518 17741 raft_consensus.cc:374] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:30.391589 17741 raft_consensus.cc:729] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc7440876978457a9a615de634184d8a, State: Initialized, Role: FOLLOWER
I20260807 15:09:30.391589 17740 raft_consensus.cc:729] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P ee36e3be2bee4d98b34305bbde38a7c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee36e3be2bee4d98b34305bbde38a7c0, State: Initialized, Role: FOLLOWER
I20260807 15:09:30.391662 17742 raft_consensus.cc:348] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P bf3cf4eb515647349b644f04c68f91e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc7440876978457a9a615de634184d8a" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 41035 } } peers { permanent_uuid: "bf3cf4eb515647349b644f04c68f91e5" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 37381 } } peers { permanent_uuid: "ee36e3be2bee4d98b34305bbde38a7c0" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37661 } }
I20260807 15:09:30.392001 17742 raft_consensus.cc:374] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P bf3cf4eb515647349b644f04c68f91e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:30.392061 17742 raft_consensus.cc:729] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P bf3cf4eb515647349b644f04c68f91e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf3cf4eb515647349b644f04c68f91e5, State: Initialized, Role: FOLLOWER
I20260807 15:09:30.392109 17741 consensus_queue.cc:260] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc7440876978457a9a615de634184d8a" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 41035 } } peers { permanent_uuid: "bf3cf4eb515647349b644f04c68f91e5" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 37381 } } peers { permanent_uuid: "ee36e3be2bee4d98b34305bbde38a7c0" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37661 } }
I20260807 15:09:30.392109 17740 consensus_queue.cc:260] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P ee36e3be2bee4d98b34305bbde38a7c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc7440876978457a9a615de634184d8a" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 41035 } } peers { permanent_uuid: "bf3cf4eb515647349b644f04c68f91e5" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 37381 } } peers { permanent_uuid: "ee36e3be2bee4d98b34305bbde38a7c0" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37661 } }
I20260807 15:09:30.392453 17742 consensus_queue.cc:260] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P bf3cf4eb515647349b644f04c68f91e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc7440876978457a9a615de634184d8a" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 41035 } } peers { permanent_uuid: "bf3cf4eb515647349b644f04c68f91e5" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 37381 } } peers { permanent_uuid: "ee36e3be2bee4d98b34305bbde38a7c0" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37661 } }
I20260807 15:09:30.393020 17345 heartbeater.cc:503] Master 127.15.226.126:46823 was elected leader, sending a full tablet report...
I20260807 15:09:30.393285 17741 ts_tablet_manager.cc:1428] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260807 15:09:30.393294 17710 heartbeater.cc:503] Master 127.15.226.126:46823 was elected leader, sending a full tablet report...
I20260807 15:09:30.394862 17742 ts_tablet_manager.cc:1428] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P bf3cf4eb515647349b644f04c68f91e5: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260807 15:09:30.395227 17530 heartbeater.cc:503] Master 127.15.226.126:46823 was elected leader, sending a full tablet report...
I20260807 15:09:30.393285 17740 ts_tablet_manager.cc:1428] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P ee36e3be2bee4d98b34305bbde38a7c0: Time spent starting tablet: real 0.008s user 0.007s sys 0.001s
I20260807 15:09:30.468694 17748 raft_consensus.cc:482] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:09:30.469033 17748 raft_consensus.cc:504] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc7440876978457a9a615de634184d8a" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 41035 } } peers { permanent_uuid: "bf3cf4eb515647349b644f04c68f91e5" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 37381 } } peers { permanent_uuid: "ee36e3be2bee4d98b34305bbde38a7c0" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37661 } }
I20260807 15:09:30.470436 17748 leader_election.cc:288] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf3cf4eb515647349b644f04c68f91e5 (127.15.226.66:37381), ee36e3be2bee4d98b34305bbde38a7c0 (127.15.226.67:37661)
I20260807 15:09:30.475011 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:30.470895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:43995 (local address 127.15.226.67:37661)
0807 15:09:30.471081 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:30.471085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:30.471364 (+ 279us) server_negotiation.cc:407] Connection header received
0807 15:09:30.471472 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:30.471478 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:30.471540 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:30.471615 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:30.472021 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.472572 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.473822 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.474297 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.474398 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:30.474470 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:30.474786 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 15:09:30.474859 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 15:09:30.475061 17752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:30.470821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:37381 (local address 127.15.226.65:34207)
0807 15:09:30.471433 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:30.471448 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:30.471551 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:30.471848 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:30.471851 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:30.471866 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:30.472002 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:30.472008 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.472756 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.472759 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:30.473688 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:30.473697 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.474536 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.474539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:30.474620 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:30.474639 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 15:09:30.474785 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 15:09:30.474859 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"mutex_wait_us":41,"thread_start_us":171,"threads_started":1}
I20260807 15:09:30.475020 17739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:30.471032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:34207 (local address 127.15.226.66:37381)
0807 15:09:30.471184 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:30.471195 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:30.471544 (+ 349us) server_negotiation.cc:407] Connection header received
0807 15:09:30.471668 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:30.471671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:30.471740 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:30.471827 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:30.472128 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.472613 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.473823 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.474386 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.474508 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:30.474584 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:30.474814 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 15:09:30.474892 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 15:09:30.475111 17753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:30.470787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:37661 (local address 127.15.226.65:43995)
0807 15:09:30.471250 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:30.471264 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:30.471379 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:30.471664 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:30.471667 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:30.471686 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:30.471882 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:30.471893 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.472701 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.472707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:30.473687 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:30.473697 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.474439 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.474446 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:30.474592 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:30.474629 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 15:09:30.474746 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 15:09:30.474848 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":26,"thread_start_us":178,"threads_started":1}
I20260807 15:09:30.476080 17648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e4f5e1dbff84fb28bbcec4e9c879d1f" candidate_uuid: "fc7440876978457a9a615de634184d8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee36e3be2bee4d98b34305bbde38a7c0" is_pre_election: true
I20260807 15:09:30.476477 17648 raft_consensus.cc:2454] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P ee36e3be2bee4d98b34305bbde38a7c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc7440876978457a9a615de634184d8a in term 0.
I20260807 15:09:30.477064 17187 leader_election.cc:302] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee36e3be2bee4d98b34305bbde38a7c0, fc7440876978457a9a615de634184d8a; no voters:
I20260807 15:09:30.477393 17748 raft_consensus.cc:2781] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:09:30.477480 17748 raft_consensus.cc:482] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:09:30.477512 17748 raft_consensus.cc:3037] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:30.478523 17463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e4f5e1dbff84fb28bbcec4e9c879d1f" candidate_uuid: "fc7440876978457a9a615de634184d8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf3cf4eb515647349b644f04c68f91e5" is_pre_election: true
I20260807 15:09:30.478737 17748 raft_consensus.cc:504] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc7440876978457a9a615de634184d8a" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 41035 } } peers { permanent_uuid: "bf3cf4eb515647349b644f04c68f91e5" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 37381 } } peers { permanent_uuid: "ee36e3be2bee4d98b34305bbde38a7c0" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37661 } }
I20260807 15:09:30.478940 17463 raft_consensus.cc:2454] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P bf3cf4eb515647349b644f04c68f91e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc7440876978457a9a615de634184d8a in term 0.
I20260807 15:09:30.479175 17748 leader_election.cc:288] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a [CANDIDATE]: Term 1 election: Requested vote from peers bf3cf4eb515647349b644f04c68f91e5 (127.15.226.66:37381), ee36e3be2bee4d98b34305bbde38a7c0 (127.15.226.67:37661)
I20260807 15:09:30.479679 17648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e4f5e1dbff84fb28bbcec4e9c879d1f" candidate_uuid: "fc7440876978457a9a615de634184d8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee36e3be2bee4d98b34305bbde38a7c0"
I20260807 15:09:30.479832 17648 raft_consensus.cc:3037] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P ee36e3be2bee4d98b34305bbde38a7c0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:30.479975 17463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e4f5e1dbff84fb28bbcec4e9c879d1f" candidate_uuid: "fc7440876978457a9a615de634184d8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf3cf4eb515647349b644f04c68f91e5"
I20260807 15:09:30.480166 17463 raft_consensus.cc:3037] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P bf3cf4eb515647349b644f04c68f91e5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:30.481284 17648 raft_consensus.cc:2454] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P ee36e3be2bee4d98b34305bbde38a7c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc7440876978457a9a615de634184d8a in term 1.
I20260807 15:09:30.481628 17187 leader_election.cc:302] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ee36e3be2bee4d98b34305bbde38a7c0, fc7440876978457a9a615de634184d8a; no voters:
I20260807 15:09:30.481844 17748 raft_consensus.cc:2781] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:30.482070 17463 raft_consensus.cc:2454] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P bf3cf4eb515647349b644f04c68f91e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc7440876978457a9a615de634184d8a in term 1.
I20260807 15:09:30.482125 17748 raft_consensus.cc:686] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a [term 1 LEADER]: Becoming Leader. State: Replica: fc7440876978457a9a615de634184d8a, State: Running, Role: LEADER
I20260807 15:09:30.482529 17748 consensus_queue.cc:237] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc7440876978457a9a615de634184d8a" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 41035 } } peers { permanent_uuid: "bf3cf4eb515647349b644f04c68f91e5" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 37381 } } peers { permanent_uuid: "ee36e3be2bee4d98b34305bbde38a7c0" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37661 } }
I20260807 15:09:30.486287 17062 catalog_manager.cc:5495] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a reported cstate change: term changed from 0 to 1, leader changed from <none> to fc7440876978457a9a615de634184d8a (127.15.226.65). New cstate: current_term: 1 leader_uuid: "fc7440876978457a9a615de634184d8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc7440876978457a9a615de634184d8a" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 41035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf3cf4eb515647349b644f04c68f91e5" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 37381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee36e3be2bee4d98b34305bbde38a7c0" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37661 } health_report { overall_health: UNKNOWN } } }
Aug 07 15:09:30 dist-test-slave-3kk6 krb5kdc[17005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.65@KRBTEST.COM
I20260807 15:09:30.525585 17733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:30.514455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44762 (local address 127.15.226.65:41035)
0807 15:09:30.514603 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:30.514610 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:30.514907 (+ 297us) server_negotiation.cc:407] Connection header received
0807 15:09:30.515078 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:30.515081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:30.515143 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:30.515239 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:30.516035 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.516579 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.517572 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.517809 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.517935 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:30.521092 (+ 3157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:30.521108 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:30.521121 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:30.521147 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:30.523358 (+ 2211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:30.523897 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:30.523905 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:30.523907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:30.524027 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:30.524393 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:30.524397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:30.524399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:30.524785 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:30.525127 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:30.525264 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 15:09:30.525405 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 15:09:30 dist-test-slave-3kk6 krb5kdc[17005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115370, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260807 15:09:30.544756 17713 tablet.cc:2369] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P ee36e3be2bee4d98b34305bbde38a7c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 15:09:30 dist-test-slave-3kk6 krb5kdc[17005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115370, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.226.65@KRBTEST.COM
I20260807 15:09:30.559563 17733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:30.550085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44776 (local address 127.15.226.65:41035)
0807 15:09:30.550269 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:30.550274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:30.550423 (+ 149us) server_negotiation.cc:407] Connection header received
0807 15:09:30.550603 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:30.550606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:30.550714 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:30.550812 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:30.551576 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.552139 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.553021 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.553273 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.553400 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:30.556167 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:30.556171 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:30.556173 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:30.556201 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:30.557615 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:30.558095 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:30.558098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:30.558100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:30.558193 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:30.558493 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:30.558496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:30.558498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:30.558774 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:30.558944 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:30.559243 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 15:09:30.559391 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260807 15:09:30.561022 17532 tablet.cc:2369] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P bf3cf4eb515647349b644f04c68f91e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:09:30.574215 17347 tablet.cc:2369] T 4e4f5e1dbff84fb28bbcec4e9c879d1f P fc7440876978457a9a615de634184d8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 15:09:30 dist-test-slave-3kk6 krb5kdc[17005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 15:09:30 dist-test-slave-3kk6 krb5kdc[17005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.65@KRBTEST.COM
I20260807 15:09:30.590711 17733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:30.581352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44792 (local address 127.15.226.65:41035)
0807 15:09:30.581538 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:30.581542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:30.581680 (+ 138us) server_negotiation.cc:407] Connection header received
0807 15:09:30.581889 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:30.581892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:30.581950 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:30.582055 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:30.582920 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.583449 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.584346 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:30.584580 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:30.584700 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:30.587393 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:30.587398 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:30.587406 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:30.587436 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:30.588805 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:30.589319 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:30.589323 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:30.589325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:30.589418 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:30.589714 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:30.589718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:30.589720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:30.589927 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:30.590098 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:30.590350 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 15:09:30.590485 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260807 15:09:30.592549 17224 tablet_service.cc:2908] Rejecting scan request for tablet 4e4f5e1dbff84fb28bbcec4e9c879d1f: Uninitialized: clean time has not yet been initialized
I20260807 15:09:30.595466 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 17162
I20260807 15:09:30.616312 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 17353
I20260807 15:09:30.629854 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 17537
I20260807 15:09:30.653760 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 17022
2026-08-07T15:09:30Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2712 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:09:30 dist-test-slave-3kk6 krb5kdc[17775](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:09:30 dist-test-slave-3kk6 krb5kdc[17775](info): set up 2 sockets
Aug 07 15:09:30 dist-test-slave-3kk6 krb5kdc[17775](info): commencing operation
krb5kdc: starting...
W20260807 15:09:31.935664 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:09:32 dist-test-slave-3kk6 krb5kdc[17775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:09:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:09:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:09:32.078291 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:37907
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:37907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:32.202909 17793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:32.203299 17793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:32.203385 17793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:32.208315 17793 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:32.208442 17793 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 15:09:32.208475 17793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:32.208510 17793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:32.208564 17793 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:32.208619 17793 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:09:32.214862 17793 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:37907
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:37907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:32.216215 17793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:32.217334 17793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:32.226740 17801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:32.226734 17804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:32.226806 17799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:32.226801 17793 server_base.cc:1034] running on GCE node
I20260807 15:09:32.227674 17793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:32.228884 17793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:32.230094 17793 hybrid_clock.cc:648] HybridClock initialized: now 1786115372230063 us; error 48 us; skew 500 ppm
Aug 07 15:09:32 dist-test-slave-3kk6 krb5kdc[17775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115372, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:32.237712 17793 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:09:32.238837 17793 webserver.cc:460] Webserver started at http://127.15.226.126:37971/ using document root <none> and password file <none>
I20260807 15:09:32.239393 17793 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:32.239454 17793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:32.239632 17793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:32.241724 17793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "58e2a93dcda74f96b2ea13ce691c6d12"
format_stamp: "Formatted at 2026-08-07 15:09:32 on dist-test-slave-3kk6"
server_key: "37427814a82b58e82568520096884d04"
server_key_iv: "1fab42159ff540dfa308da1835fd9e5d"
server_key_version: "encryptionkey@0"
I20260807 15:09:32.242264 17793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "58e2a93dcda74f96b2ea13ce691c6d12"
format_stamp: "Formatted at 2026-08-07 15:09:32 on dist-test-slave-3kk6"
server_key: "37427814a82b58e82568520096884d04"
server_key_iv: "1fab42159ff540dfa308da1835fd9e5d"
server_key_version: "encryptionkey@0"
I20260807 15:09:32.246331 17793 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260807 15:09:32.249040 17815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:32.250459 17793 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:09:32.250608 17793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "58e2a93dcda74f96b2ea13ce691c6d12"
format_stamp: "Formatted at 2026-08-07 15:09:32 on dist-test-slave-3kk6"
server_key: "37427814a82b58e82568520096884d04"
server_key_iv: "1fab42159ff540dfa308da1835fd9e5d"
server_key_version: "encryptionkey@0"
I20260807 15:09:32.250790 17793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:32.264827 17793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:32.268429 17793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:32.268651 17793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:32.277664 17793 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:37907
I20260807 15:09:32.277781 17905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:37907 every 8 connection(s)
I20260807 15:09:32.278780 17793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:09:32.281934 17906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:32.287323 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 17793
I20260807 15:09:32.287442 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:09:32.287719 16265 external_mini_cluster.cc:1428] Setting key 1d68523e820172c20f42782abca2672e
I20260807 15:09:32.288435 17906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12: Bootstrap starting.
I20260807 15:09:32.291241 17906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:32.292352 17906 log.cc:824] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:32.294880 17906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12: No bootstrap required, opened a new log
Aug 07 15:09:32 dist-test-slave-3kk6 krb5kdc[17775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:32.299633 17906 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58e2a93dcda74f96b2ea13ce691c6d12" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37907 } }
I20260807 15:09:32.299909 17906 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:32.299993 17906 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58e2a93dcda74f96b2ea13ce691c6d12, State: Initialized, Role: FOLLOWER
I20260807 15:09:32.300549 17906 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58e2a93dcda74f96b2ea13ce691c6d12" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37907 } }
I20260807 15:09:32.300722 17906 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:09:32.300824 17906 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:09:32.300957 17906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:32.302309 17906 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58e2a93dcda74f96b2ea13ce691c6d12" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37907 } }
I20260807 15:09:32.302785 17906 leader_election.cc:302] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58e2a93dcda74f96b2ea13ce691c6d12; no voters:
I20260807 15:09:32.303221 17906 leader_election.cc:288] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:09:32.303406 17912 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:32.303707 17912 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12 [term 1 LEADER]: Becoming Leader. State: Replica: 58e2a93dcda74f96b2ea13ce691c6d12, State: Running, Role: LEADER
I20260807 15:09:32.304098 17912 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58e2a93dcda74f96b2ea13ce691c6d12" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37907 } }
I20260807 15:09:32.304710 17906 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:32.305351 17912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58e2a93dcda74f96b2ea13ce691c6d12. Latest consensus state: current_term: 1 leader_uuid: "58e2a93dcda74f96b2ea13ce691c6d12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58e2a93dcda74f96b2ea13ce691c6d12" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37907 } } }
I20260807 15:09:32.305611 17912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:32.305449 17909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:32.289324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55448 (local address 127.15.226.126:37907)
0807 15:09:32.289906 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:32.289923 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:32.289961 (+ 38us) server_negotiation.cc:407] Connection header received
0807 15:09:32.290802 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:32.290823 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:32.291178 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:32.291550 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:32.292599 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:32.293691 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:32.294987 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:32.295514 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:32.295650 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:32.298901 (+ 3251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:32.298907 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:32.298932 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:32.298965 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:32.301196 (+ 2231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:32.301779 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:32.301784 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:32.301790 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:32.301940 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:32.302287 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:32.302290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:32.302291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:32.302959 (+ 668us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:32.303151 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:32.304048 (+ 897us) server_negotiation.cc:299] Negotiation successful
0807 15:09:32.304391 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":383,"thread_start_us":197,"threads_started":1}
I20260807 15:09:32.305823 17916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58e2a93dcda74f96b2ea13ce691c6d12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58e2a93dcda74f96b2ea13ce691c6d12" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37907 } } }
I20260807 15:09:32.305927 17916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:32.306231 17926 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:09:32.309774 17926 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:09:32.316478 17926 catalog_manager.cc:1347] Generated new cluster ID: 7bca5cca6eb24801b1dce031aa190ea9
I20260807 15:09:32.316589 17926 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:09:32.330238 17926 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:09:32.331794 17926 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:09:32.341732 17926 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12: Generated new TSK 0
I20260807 15:09:32.342756 17926 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.226.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
I20260807 15:09:32.385897 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:37907
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:32.513484 17941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:32.513870 17941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:32.513922 17941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:32.518162 17941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:32.518260 17941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:32.518303 17941 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:09:32.518337 17941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:09:32.524320 17941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.226.126:37907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:32.525529 17941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:32.526620 17941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:32.537343 17950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:32.537354 17954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:32.537509 17941 server_base.cc:1034] running on GCE node
W20260807 15:09:32.537362 17951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:32.538233 17941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:32.539192 17941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:32.540417 17941 hybrid_clock.cc:648] HybridClock initialized: now 1786115372540380 us; error 57 us; skew 500 ppm
Aug 07 15:09:32 dist-test-slave-3kk6 krb5kdc[17775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115372, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:32.543730 17941 init.cc:377] Logged in from keytab as kudu/127.15.226.65@KRBTEST.COM (short username kudu)
I20260807 15:09:32.544842 17941 webserver.cc:460] Webserver started at http://127.15.226.65:39123/ using document root <none> and password file <none>
I20260807 15:09:32.545485 17941 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:32.545588 17941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:32.545858 17941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:32.548296 17941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "81140df0b6cf4beb95c4bccc5b36966b"
format_stamp: "Formatted at 2026-08-07 15:09:32 on dist-test-slave-3kk6"
server_key: "a45e18773d1c86c77e574dae562177e1"
server_key_iv: "1710577f0de3bcbd46d446ffbde278cb"
server_key_version: "encryptionkey@0"
I20260807 15:09:32.548956 17941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "81140df0b6cf4beb95c4bccc5b36966b"
format_stamp: "Formatted at 2026-08-07 15:09:32 on dist-test-slave-3kk6"
server_key: "a45e18773d1c86c77e574dae562177e1"
server_key_iv: "1710577f0de3bcbd46d446ffbde278cb"
server_key_version: "encryptionkey@0"
I20260807 15:09:32.553432 17941 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 15:09:32.556891 17961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:32.558359 17941 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:09:32.558550 17941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "81140df0b6cf4beb95c4bccc5b36966b"
format_stamp: "Formatted at 2026-08-07 15:09:32 on dist-test-slave-3kk6"
server_key: "a45e18773d1c86c77e574dae562177e1"
server_key_iv: "1710577f0de3bcbd46d446ffbde278cb"
server_key_version: "encryptionkey@0"
I20260807 15:09:32.558758 17941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:32.581312 17941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:32.585449 17941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:32.585713 17941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:32.587553 17941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:32.587639 17941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:32.587721 17941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:32.587767 17941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:09:32 dist-test-slave-3kk6 krb5kdc[17775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115372, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:32.600063 17941 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:38835
I20260807 15:09:32.600896 18128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:38835 every 8 connection(s)
I20260807 15:09:32.601161 17941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:09:32.603117 17909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:32.588564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:57169 (local address 127.15.226.126:37907)
0807 15:09:32.588755 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:32.588762 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:32.589702 (+ 940us) server_negotiation.cc:407] Connection header received
0807 15:09:32.590928 (+ 1226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:32.590934 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:32.591004 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:32.591097 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:32.592618 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:32.593216 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:32.594530 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:32.594958 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:32.595081 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:32.598816 (+ 3735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:32.598821 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:32.598824 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:32.598869 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:32.600624 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:32.601303 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:32.601307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:32.601309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:32.601407 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:32.601852 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:32.601856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:32.601857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:32.602145 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:32.602256 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:32.602832 (+ 576us) server_negotiation.cc:299] Negotiation successful
0807 15:09:32.602966 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 15:09:32.603971 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:32.589027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:37907 (local address 127.15.226.65:57169)
0807 15:09:32.589505 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:32.589561 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:32.590600 (+ 1039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:32.591536 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:32.591542 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:32.591940 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:32.592444 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:32.592458 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:32.593369 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:32.593373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:32.594387 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:32.594398 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:32.595205 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:32.595214 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:32.595510 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:32.596381 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:32.596413 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:32.598556 (+ 2143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:32.600777 (+ 2221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:32.600785 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:32.600802 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:32.601187 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:32.601510 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:32.601514 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:32.601516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:32.601727 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:32.602265 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:32.602270 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:32.602534 (+ 264us) client_negotiation.cc:769] Sending connection context
0807 15:09:32.602787 (+ 253us) client_negotiation.cc:240] Negotiation successful
0807 15:09:32.603082 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":140,"threads_started":1}
I20260807 15:09:32.605551 18129 heartbeater.cc:347] Connected to a master server at 127.15.226.126:37907
I20260807 15:09:32.605736 18129 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:32.605726 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 17941
I20260807 15:09:32.605855 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:09:32.606133 16265 external_mini_cluster.cc:1428] Setting key 8e74325d1736aced547d67847c0b5dcb
I20260807 15:09:32.606469 18129 heartbeater.cc:511] Master 127.15.226.126:37907 requested a full tablet report, sending...
I20260807 15:09:32.608170 17829 ts_manager.cc:194] Registered new tserver with Master: 81140df0b6cf4beb95c4bccc5b36966b (127.15.226.65:38835)
I20260807 15:09:32.609820 17829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.65@KRBTEST.COM'} at 127.15.226.65:57169
WARNING: no policy specified for kudu/127.15.226.66@KRBTEST.COM; defaulting to no policy
I20260807 15:09:32.619462 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:32.610880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:37907 (local address 127.15.226.65:55299)
0807 15:09:32.611212 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:32.611230 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:32.611344 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:32.611627 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:32.611630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:32.611917 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:32.612043 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:32.612051 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:32.613211 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:32.613214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:32.613789 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:32.613832 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:32.614508 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:32.614511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:32.614589 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:32.615285 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:32.615298 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:32.615655 (+ 357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:32.617747 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:32.617750 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:32.617751 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:32.617969 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:32.618300 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:32.618303 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:32.618305 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:32.618386 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:32.618896 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:32.618898 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:32.618982 (+ 84us) client_negotiation.cc:769] Sending connection context
0807 15:09:32.619070 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 15:09:32.619225 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":123,"threads_started":1}
I20260807 15:09:32.619541 17909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:32.610964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:55299 (local address 127.15.226.126:37907)
0807 15:09:32.611094 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:32.611099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:32.611323 (+ 224us) server_negotiation.cc:407] Connection header received
0807 15:09:32.611433 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:32.611438 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:32.611492 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:32.611586 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:32.612225 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:32.613063 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:32.613958 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:32.614378 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:32.614504 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:32.615817 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:32.615822 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:32.615828 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:32.615865 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:32.617615 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:32.618080 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:32.618087 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:32.618091 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:32.618194 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:32.618490 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:32.618494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:32.618497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:32.618787 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:32.618898 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:32.619171 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 15:09:32.619310 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
I20260807 15:09:32.640606 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:37907
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:32.764729 18143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:32.765113 18143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:32.765196 18143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:32.769564 18143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:32.769685 18143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:32.769765 18143 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:09:32.769853 18143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:09:32.775789 18143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.226.126:37907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:32.777067 18143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:32.778205 18143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:32.789008 18154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:32.788990 18151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:32.788969 18152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:32.789502 18143 server_base.cc:1034] running on GCE node
I20260807 15:09:32.789951 18143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:32.790736 18143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:32.791945 18143 hybrid_clock.cc:648] HybridClock initialized: now 1786115372791924 us; error 38 us; skew 500 ppm
Aug 07 15:09:32 dist-test-slave-3kk6 krb5kdc[17775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115372, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:32.794986 18143 init.cc:377] Logged in from keytab as kudu/127.15.226.66@KRBTEST.COM (short username kudu)
I20260807 15:09:32.796008 18143 webserver.cc:460] Webserver started at http://127.15.226.66:40999/ using document root <none> and password file <none>
I20260807 15:09:32.796648 18143 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:32.796751 18143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:32.797003 18143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:32.799419 18143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "d46692444a974d3990ff77fd1646408d"
format_stamp: "Formatted at 2026-08-07 15:09:32 on dist-test-slave-3kk6"
server_key: "69a1c5cbaad7dd933cda342145dba062"
server_key_iv: "c41466e38b418d05130089a960fd5da3"
server_key_version: "encryptionkey@0"
I20260807 15:09:32.800065 18143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "d46692444a974d3990ff77fd1646408d"
format_stamp: "Formatted at 2026-08-07 15:09:32 on dist-test-slave-3kk6"
server_key: "69a1c5cbaad7dd933cda342145dba062"
server_key_iv: "c41466e38b418d05130089a960fd5da3"
server_key_version: "encryptionkey@0"
I20260807 15:09:32.804461 18143 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:09:32.807840 18165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:32.809410 18143 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:09:32.809556 18143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "d46692444a974d3990ff77fd1646408d"
format_stamp: "Formatted at 2026-08-07 15:09:32 on dist-test-slave-3kk6"
server_key: "69a1c5cbaad7dd933cda342145dba062"
server_key_iv: "c41466e38b418d05130089a960fd5da3"
server_key_version: "encryptionkey@0"
I20260807 15:09:32.809674 18143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:32.841687 18143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:32.845475 18143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:32.845732 18143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:32.847470 18143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:32.847553 18143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:32.847633 18143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:32.847678 18143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:09:32 dist-test-slave-3kk6 krb5kdc[17775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115372, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:32.859622 18143 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:44589
I20260807 15:09:32.859778 18326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:44589 every 8 connection(s)
I20260807 15:09:32.860769 18143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:09:32.866581 17909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:32.848543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:52195 (local address 127.15.226.126:37907)
0807 15:09:32.848687 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:32.848690 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:32.849936 (+ 1246us) server_negotiation.cc:407] Connection header received
0807 15:09:32.851276 (+ 1340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:32.851280 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:32.851344 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:32.851459 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:32.853171 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:32.854092 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:32.856104 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:32.856456 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:32.856598 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:32.860010 (+ 3412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:32.860014 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:32.860017 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:32.860057 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:32.862085 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:32.864771 (+ 2686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:32.864774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:32.864775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:32.864887 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:32.865323 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:32.865326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:32.865328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:32.865556 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:32.865677 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:32.866241 (+ 564us) server_negotiation.cc:299] Negotiation successful
0807 15:09:32.866391 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:09:32.867462 18174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:32.849129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:37907 (local address 127.15.226.66:52195)
0807 15:09:32.849757 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:32.849807 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:32.850997 (+ 1190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:32.851658 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:32.851669 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:32.852194 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:32.852994 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:32.853020 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:32.854256 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:32.854260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:32.855895 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:32.855908 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:32.856634 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:32.856637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:32.856846 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:32.857673 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:32.857699 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:32.859807 (+ 2108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:32.862237 (+ 2430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:32.862251 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:32.862268 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:32.862564 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:32.865004 (+ 2440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:32.865007 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:32.865010 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:32.865194 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:32.865677 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:32.865686 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:32.865975 (+ 289us) client_negotiation.cc:769] Sending connection context
0807 15:09:32.866179 (+ 204us) client_negotiation.cc:240] Negotiation successful
0807 15:09:32.866556 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":161,"threads_started":1}
I20260807 15:09:32.868814 18327 heartbeater.cc:347] Connected to a master server at 127.15.226.126:37907
I20260807 15:09:32.868979 18327 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:32.869639 18327 heartbeater.cc:511] Master 127.15.226.126:37907 requested a full tablet report, sending...
I20260807 15:09:32.870150 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 18143
I20260807 15:09:32.870288 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:09:32.870700 16265 external_mini_cluster.cc:1428] Setting key 438befe180fdf7b916f01e0b6ff18a48
I20260807 15:09:32.871450 17834 ts_manager.cc:194] Registered new tserver with Master: d46692444a974d3990ff77fd1646408d (127.15.226.66:44589)
I20260807 15:09:32.872285 17834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.66@KRBTEST.COM'} at 127.15.226.66:52195
I20260807 15:09:32.880967 18334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:32.871518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:37907 (local address 127.15.226.66:41573)
0807 15:09:32.871867 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:32.871882 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:32.871997 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:32.872744 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:32.872746 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:32.872957 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:32.873096 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:32.873103 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:32.873966 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:32.873970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:32.874861 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:32.874878 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:32.875372 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:32.875377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:32.875515 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:32.876120 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:32.876143 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:32.876712 (+ 569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:32.879027 (+ 2315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:32.879030 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:32.879031 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:32.879252 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:32.879627 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:32.879629 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:32.879631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:32.879719 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:32.880449 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:32.880451 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:32.880541 (+ 90us) client_negotiation.cc:769] Sending connection context
0807 15:09:32.880627 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 15:09:32.880757 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":4,"thread_start_us":138,"threads_started":1}
I20260807 15:09:32.881109 17909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:32.872021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:41573 (local address 127.15.226.126:37907)
0807 15:09:32.872134 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:32.872138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:32.872154 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:09:32.872193 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:32.872196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:32.872238 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:32.872349 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:32.873261 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:32.873819 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:32.875024 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:32.875241 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:32.875330 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:32.876871 (+ 1541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:32.876874 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:32.876876 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:32.876900 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:32.878872 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:32.879371 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:32.879377 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:32.879383 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:32.879500 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:32.879907 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:32.879914 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:32.879918 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:32.880320 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:32.880457 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:32.880732 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 15:09:32.880902 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
WARNING: no policy specified for kudu/127.15.226.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
I20260807 15:09:32.906476 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:37907
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:33.029246 18338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:33.029604 18338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:33.029686 18338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:33.033828 18338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:33.033949 18338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:33.034026 18338 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:09:33.034112 18338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:09:33.040727 18338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.226.126:37907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:33.041945 18338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:33.043084 18338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:33.054129 18347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:33.054086 18343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:33.054534 18338 server_base.cc:1034] running on GCE node
W20260807 15:09:33.054080 18345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:33.055197 18338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:33.055876 18338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:33.057049 18338 hybrid_clock.cc:648] HybridClock initialized: now 1786115373057028 us; error 38 us; skew 500 ppm
Aug 07 15:09:33 dist-test-slave-3kk6 krb5kdc[17775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115373, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:33.060263 18338 init.cc:377] Logged in from keytab as kudu/127.15.226.67@KRBTEST.COM (short username kudu)
I20260807 15:09:33.061316 18338 webserver.cc:460] Webserver started at http://127.15.226.67:38235/ using document root <none> and password file <none>
I20260807 15:09:33.061944 18338 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:33.062016 18338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:33.062326 18338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:33.064770 18338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "e16f82b89e3a4e0ab2ccdba725cd36da"
format_stamp: "Formatted at 2026-08-07 15:09:33 on dist-test-slave-3kk6"
server_key: "ee6a47c48ff3592d6663b1db08663885"
server_key_iv: "caf5f2bc581cc6861d7e2daf1a547130"
server_key_version: "encryptionkey@0"
I20260807 15:09:33.065419 18338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "e16f82b89e3a4e0ab2ccdba725cd36da"
format_stamp: "Formatted at 2026-08-07 15:09:33 on dist-test-slave-3kk6"
server_key: "ee6a47c48ff3592d6663b1db08663885"
server_key_iv: "caf5f2bc581cc6861d7e2daf1a547130"
server_key_version: "encryptionkey@0"
I20260807 15:09:33.069628 18338 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:09:33.072577 18359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:33.073951 18338 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:09:33.074091 18338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "e16f82b89e3a4e0ab2ccdba725cd36da"
format_stamp: "Formatted at 2026-08-07 15:09:33 on dist-test-slave-3kk6"
server_key: "ee6a47c48ff3592d6663b1db08663885"
server_key_iv: "caf5f2bc581cc6861d7e2daf1a547130"
server_key_version: "encryptionkey@0"
I20260807 15:09:33.074211 18338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:33.102583 18338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:33.106431 18338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:33.106711 18338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:33.108373 18338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:33.108430 18338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:33.108476 18338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:33.108531 18338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:09:33 dist-test-slave-3kk6 krb5kdc[17775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115373, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:33.121445 18338 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:41249
I20260807 15:09:33.121542 18531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:41249 every 8 connection(s)
I20260807 15:09:33.122830 18338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:09:33.125648 17909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.109398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:47153 (local address 127.15.226.126:37907)
0807 15:09:33.109538 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:33.109542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:33.110884 (+ 1342us) server_negotiation.cc:407] Connection header received
0807 15:09:33.112198 (+ 1314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:33.112202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:33.112275 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:33.112397 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:33.114379 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.115357 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.117159 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.117415 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.117547 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.121150 (+ 3603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:33.121155 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:33.121158 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:33.121197 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:33.123490 (+ 2293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:33.124122 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:33.124125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:33.124126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:33.124234 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:33.124584 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:33.124586 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:33.124587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:33.124808 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:33.124897 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:33.125395 (+ 498us) server_negotiation.cc:299] Negotiation successful
0807 15:09:33.125517 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 15:09:33.126390 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 18338
I20260807 15:09:33.126526 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:09:33.126845 16265 external_mini_cluster.cc:1428] Setting key c4406deea5d973074c499bf1224c12af
I20260807 15:09:33.126905 18371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.110006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:37907 (local address 127.15.226.67:47153)
0807 15:09:33.110578 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:33.110626 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:33.111854 (+ 1228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:33.112653 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:33.112665 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:33.113227 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:33.114106 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.114135 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.115573 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.115578 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.116975 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.116992 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.117618 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.117622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.117827 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.118734 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:33.118754 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:33.120892 (+ 2138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:33.123669 (+ 2777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:33.123675 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:33.123690 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:33.124000 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:33.124340 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:33.124342 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:33.124344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:33.124491 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:33.124910 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:33.124915 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:33.125167 (+ 252us) client_negotiation.cc:769] Sending connection context
0807 15:09:33.125332 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 15:09:33.125615 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":154,"threads_started":1}
I20260807 15:09:33.128454 18532 heartbeater.cc:347] Connected to a master server at 127.15.226.126:37907
I20260807 15:09:33.128646 18532 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:33.129336 18532 heartbeater.cc:511] Master 127.15.226.126:37907 requested a full tablet report, sending...
I20260807 15:09:33.130491 17833 ts_manager.cc:194] Registered new tserver with Master: e16f82b89e3a4e0ab2ccdba725cd36da (127.15.226.67:41249)
I20260807 15:09:33.131374 17833 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.67@KRBTEST.COM'} at 127.15.226.67:47153
I20260807 15:09:33.138595 18542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.130949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:37907 (local address 127.15.226.67:33739)
0807 15:09:33.131262 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:33.131279 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:33.131379 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:33.132046 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:33.132050 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:33.132283 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:33.132487 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.132494 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.133496 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.133498 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.134061 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.134068 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.134506 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.134508 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.134579 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.135284 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:33.135297 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:33.135650 (+ 353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:33.137260 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:33.137262 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:33.137264 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:33.137474 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:33.137726 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:33.137728 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:33.137730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:33.137807 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:33.138148 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:33.138150 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:33.138232 (+ 82us) client_negotiation.cc:769] Sending connection context
0807 15:09:33.138314 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 15:09:33.138436 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":107,"threads_started":1}
I20260807 15:09:33.138602 17909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.131677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:33739 (local address 127.15.226.126:37907)
0807 15:09:33.131835 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:33.131838 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:33.131850 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:09:33.131883 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:33.131885 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:33.131920 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:33.131988 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:33.132620 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.133355 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.134172 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.134395 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.134480 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.135798 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:33.135801 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:33.135804 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:33.135827 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:33.137149 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:33.137561 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:33.137564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:33.137565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:33.137644 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:33.137886 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:33.137889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:33.137891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:33.138066 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:33.138143 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:33.138379 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 15:09:33.138482 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 15:09:33.141671 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 15:09:33 dist-test-slave-3kk6 krb5kdc[17775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115373, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 15:09:33 dist-test-slave-3kk6 krb5kdc[17775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115373, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:33.170822 17909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.161734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55464 (local address 127.15.226.126:37907)
0807 15:09:33.161913 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:33.161916 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:33.161929 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:09:33.161973 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:33.161976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:33.162017 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:33.162119 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:33.163024 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.163586 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.164462 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.164690 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.164799 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.167438 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:33.167442 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:33.167444 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:33.167471 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:33.169016 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:33.169484 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:33.169487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:33.169488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:33.169569 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:33.169881 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:33.169888 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:33.169890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:33.170076 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:33.170218 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:33.170482 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 15:09:33.170610 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 15:09:33.174033 17833 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55464:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:09:33.177217 17833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:09:33.193589 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.187384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50998 (local address 127.15.226.65:38835)
0807 15:09:33.187731 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:33.187739 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:33.187868 (+ 129us) server_negotiation.cc:407] Connection header received
0807 15:09:33.188291 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:33.188298 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:33.188533 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:33.188710 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:33.189042 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.190119 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.192005 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.192852 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.192963 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.193122 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:33.193331 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 15:09:33.193440 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":102,"threads_started":1}
I20260807 15:09:33.193601 18554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.187237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:38835 (local address 127.0.0.1:50998)
0807 15:09:33.187684 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:33.187761 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:33.187925 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:33.188700 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:33.188706 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:33.188731 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:33.188916 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.188925 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.190239 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.190244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.191842 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.191854 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.192987 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.192991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.193131 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.193179 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 15:09:33.193297 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 15:09:33.193429 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":8,"thread_start_us":122,"threads_started":1}
I20260807 15:09:33.193868 18558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.188313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48726 (local address 127.15.226.66:44589)
0807 15:09:33.189373 (+ 1060us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:33.189383 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:33.189403 (+ 20us) server_negotiation.cc:407] Connection header received
0807 15:09:33.189464 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:33.189468 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:33.189603 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:33.189723 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:33.190018 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.191032 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.192374 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.192974 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.193407 (+ 433us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.193572 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:33.193626 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 15:09:33.193723 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":959,"thread_start_us":116,"threads_started":1}
I20260807 15:09:33.194160 18556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.188100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:44589 (local address 127.0.0.1:48726)
0807 15:09:33.188381 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:33.188399 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:33.188497 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:33.189743 (+ 1246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:33.189746 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:33.189766 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:33.189876 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.189882 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.191147 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.191153 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.192245 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.192255 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.193112 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.193115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.193186 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.193204 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 15:09:33.193281 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 15:09:33.194020 (+ 739us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":628,"thread_start_us":118,"threads_started":1}
I20260807 15:09:33.194680 18561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.188896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51022 (local address 127.15.226.67:41249)
0807 15:09:33.189433 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:33.189443 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:33.189950 (+ 507us) server_negotiation.cc:407] Connection header received
0807 15:09:33.190076 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:33.190087 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:33.190267 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:33.190423 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:33.190714 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.191888 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.193210 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.193989 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.194106 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.194268 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:33.194423 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 15:09:33.194511 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":133,"threads_started":1}
I20260807 15:09:33.194576 18560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.188478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:41249 (local address 127.0.0.1:51022)
0807 15:09:33.189829 (+ 1351us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:33.189844 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:33.189961 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:33.190422 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:33.190425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:33.190445 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:33.190568 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.190574 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.192014 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.192017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.193104 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.193114 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.194134 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.194138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.194247 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.194296 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 15:09:33.194388 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 15:09:33.194462 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":37,"thread_start_us":97,"threads_started":1}
I20260807 15:09:33.197074 18230 tablet_service.cc:1467] Processing CreateTablet for tablet b5c004fc3e7f47bd9001b84a954b2a73 (DEFAULT_TABLE table=test-table [id=2bf252edcace414abfc71d42f8904716]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:33.197100 18033 tablet_service.cc:1467] Processing CreateTablet for tablet b5c004fc3e7f47bd9001b84a954b2a73 (DEFAULT_TABLE table=test-table [id=2bf252edcace414abfc71d42f8904716]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:33.198086 18230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5c004fc3e7f47bd9001b84a954b2a73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:33.198145 18033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5c004fc3e7f47bd9001b84a954b2a73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:33.197952 18433 tablet_service.cc:1467] Processing CreateTablet for tablet b5c004fc3e7f47bd9001b84a954b2a73 (DEFAULT_TABLE table=test-table [id=2bf252edcace414abfc71d42f8904716]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:33.198982 18433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5c004fc3e7f47bd9001b84a954b2a73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:33.205657 18563 tablet_bootstrap.cc:492] T b5c004fc3e7f47bd9001b84a954b2a73 P d46692444a974d3990ff77fd1646408d: Bootstrap starting.
I20260807 15:09:33.206991 18566 tablet_bootstrap.cc:492] T b5c004fc3e7f47bd9001b84a954b2a73 P e16f82b89e3a4e0ab2ccdba725cd36da: Bootstrap starting.
I20260807 15:09:33.207305 18564 tablet_bootstrap.cc:492] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b: Bootstrap starting.
I20260807 15:09:33.208874 18563 tablet_bootstrap.cc:654] T b5c004fc3e7f47bd9001b84a954b2a73 P d46692444a974d3990ff77fd1646408d: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:33.209499 18566 tablet_bootstrap.cc:654] T b5c004fc3e7f47bd9001b84a954b2a73 P e16f82b89e3a4e0ab2ccdba725cd36da: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:33.209547 18564 tablet_bootstrap.cc:654] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:33.210566 18564 log.cc:824] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:33.210566 18566 log.cc:824] T b5c004fc3e7f47bd9001b84a954b2a73 P e16f82b89e3a4e0ab2ccdba725cd36da: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:33.211182 18563 log.cc:824] T b5c004fc3e7f47bd9001b84a954b2a73 P d46692444a974d3990ff77fd1646408d: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:33.213128 18564 tablet_bootstrap.cc:492] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b: No bootstrap required, opened a new log
I20260807 15:09:33.213389 18564 ts_tablet_manager.cc:1397] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 15:09:33.213702 18566 tablet_bootstrap.cc:492] T b5c004fc3e7f47bd9001b84a954b2a73 P e16f82b89e3a4e0ab2ccdba725cd36da: No bootstrap required, opened a new log
I20260807 15:09:33.213914 18566 ts_tablet_manager.cc:1397] T b5c004fc3e7f47bd9001b84a954b2a73 P e16f82b89e3a4e0ab2ccdba725cd36da: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 15:09:33.214895 18563 tablet_bootstrap.cc:492] T b5c004fc3e7f47bd9001b84a954b2a73 P d46692444a974d3990ff77fd1646408d: No bootstrap required, opened a new log
I20260807 15:09:33.215207 18563 ts_tablet_manager.cc:1397] T b5c004fc3e7f47bd9001b84a954b2a73 P d46692444a974d3990ff77fd1646408d: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260807 15:09:33.218228 18566 raft_consensus.cc:348] T b5c004fc3e7f47bd9001b84a954b2a73 P e16f82b89e3a4e0ab2ccdba725cd36da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.218940 18566 raft_consensus.cc:374] T b5c004fc3e7f47bd9001b84a954b2a73 P e16f82b89e3a4e0ab2ccdba725cd36da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:33.219090 18566 raft_consensus.cc:729] T b5c004fc3e7f47bd9001b84a954b2a73 P e16f82b89e3a4e0ab2ccdba725cd36da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e16f82b89e3a4e0ab2ccdba725cd36da, State: Initialized, Role: FOLLOWER
I20260807 15:09:33.219722 18566 consensus_queue.cc:260] T b5c004fc3e7f47bd9001b84a954b2a73 P e16f82b89e3a4e0ab2ccdba725cd36da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.219708 18564 raft_consensus.cc:348] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.220021 18564 raft_consensus.cc:374] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:33.220091 18564 raft_consensus.cc:729] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81140df0b6cf4beb95c4bccc5b36966b, State: Initialized, Role: FOLLOWER
I20260807 15:09:33.220602 18564 consensus_queue.cc:260] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.220799 18532 heartbeater.cc:503] Master 127.15.226.126:37907 was elected leader, sending a full tablet report...
I20260807 15:09:33.221293 18129 heartbeater.cc:503] Master 127.15.226.126:37907 was elected leader, sending a full tablet report...
I20260807 15:09:33.221796 18564 ts_tablet_manager.cc:1428] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260807 15:09:33.222420 18566 ts_tablet_manager.cc:1428] T b5c004fc3e7f47bd9001b84a954b2a73 P e16f82b89e3a4e0ab2ccdba725cd36da: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260807 15:09:33.225617 18563 raft_consensus.cc:348] T b5c004fc3e7f47bd9001b84a954b2a73 P d46692444a974d3990ff77fd1646408d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.226253 18563 raft_consensus.cc:374] T b5c004fc3e7f47bd9001b84a954b2a73 P d46692444a974d3990ff77fd1646408d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:33.226331 18563 raft_consensus.cc:729] T b5c004fc3e7f47bd9001b84a954b2a73 P d46692444a974d3990ff77fd1646408d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d46692444a974d3990ff77fd1646408d, State: Initialized, Role: FOLLOWER
I20260807 15:09:33.226869 18563 consensus_queue.cc:260] T b5c004fc3e7f47bd9001b84a954b2a73 P d46692444a974d3990ff77fd1646408d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.227622 18327 heartbeater.cc:503] Master 127.15.226.126:37907 was elected leader, sending a full tablet report...
I20260807 15:09:33.227998 18563 ts_tablet_manager.cc:1428] T b5c004fc3e7f47bd9001b84a954b2a73 P d46692444a974d3990ff77fd1646408d: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
W20260807 15:09:33.352774 18131 tablet.cc:2369] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:09:33.362221 18328 tablet.cc:2369] T b5c004fc3e7f47bd9001b84a954b2a73 P d46692444a974d3990ff77fd1646408d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:09:33.374058 18535 tablet.cc:2369] T b5c004fc3e7f47bd9001b84a954b2a73 P e16f82b89e3a4e0ab2ccdba725cd36da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:09:33.458393 18576 raft_consensus.cc:482] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:09:33.458772 18576 raft_consensus.cc:504] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.460191 18576 leader_election.cc:288] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d46692444a974d3990ff77fd1646408d (127.15.226.66:44589), e16f82b89e3a4e0ab2ccdba725cd36da (127.15.226.67:41249)
I20260807 15:09:33.465310 18558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.460624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:45817 (local address 127.15.226.66:44589)
0807 15:09:33.460776 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:33.460780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:33.460958 (+ 178us) server_negotiation.cc:407] Connection header received
0807 15:09:33.461228 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:33.461232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:33.461292 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:33.461410 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:33.461860 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.462522 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.464036 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.464587 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.464687 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.464733 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:33.465073 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 15:09:33.465156 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:09:33.465348 18578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.460478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:44589 (local address 127.15.226.65:45817)
0807 15:09:33.460832 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:33.460851 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:33.460984 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:33.461478 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:33.461480 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:33.461499 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:33.461706 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.461717 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.462632 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.462635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.463901 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.463912 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.464714 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.464718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.464893 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.464925 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 15:09:33.465021 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 15:09:33.465101 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":155,"threads_started":1}
I20260807 15:09:33.465669 18561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.461132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:58523 (local address 127.15.226.67:41249)
0807 15:09:33.461335 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:33.461340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:33.461673 (+ 333us) server_negotiation.cc:407] Connection header received
0807 15:09:33.461738 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:33.461744 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:33.461805 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:33.461880 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:33.462151 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.462977 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.464417 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.464969 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.465088 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.465135 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:33.465413 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 15:09:33.465507 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 15:09:33.465917 18579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.461142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:41249 (local address 127.15.226.65:58523)
0807 15:09:33.461553 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:33.461566 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:33.461656 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:33.461894 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:33.461897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:33.461914 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:33.462031 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.462037 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.463114 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.463116 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.464278 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.464289 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.465097 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.465100 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.465243 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.465275 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 15:09:33.465361 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 15:09:33.465591 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":146,"threads_started":1}
I20260807 15:09:33.466331 18257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5c004fc3e7f47bd9001b84a954b2a73" candidate_uuid: "81140df0b6cf4beb95c4bccc5b36966b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d46692444a974d3990ff77fd1646408d" is_pre_election: true
I20260807 15:09:33.466624 18466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5c004fc3e7f47bd9001b84a954b2a73" candidate_uuid: "81140df0b6cf4beb95c4bccc5b36966b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" is_pre_election: true
I20260807 15:09:33.466748 18257 raft_consensus.cc:2454] T b5c004fc3e7f47bd9001b84a954b2a73 P d46692444a974d3990ff77fd1646408d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81140df0b6cf4beb95c4bccc5b36966b in term 0.
I20260807 15:09:33.467003 18466 raft_consensus.cc:2454] T b5c004fc3e7f47bd9001b84a954b2a73 P e16f82b89e3a4e0ab2ccdba725cd36da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81140df0b6cf4beb95c4bccc5b36966b in term 0.
I20260807 15:09:33.467483 17968 leader_election.cc:302] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81140df0b6cf4beb95c4bccc5b36966b, d46692444a974d3990ff77fd1646408d; no voters:
I20260807 15:09:33.467758 18576 raft_consensus.cc:2781] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:09:33.467862 18576 raft_consensus.cc:482] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:09:33.467921 18576 raft_consensus.cc:3037] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:33.469089 18576 raft_consensus.cc:504] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.469473 18576 leader_election.cc:288] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b [CANDIDATE]: Term 1 election: Requested vote from peers d46692444a974d3990ff77fd1646408d (127.15.226.66:44589), e16f82b89e3a4e0ab2ccdba725cd36da (127.15.226.67:41249)
I20260807 15:09:33.469892 18257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5c004fc3e7f47bd9001b84a954b2a73" candidate_uuid: "81140df0b6cf4beb95c4bccc5b36966b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d46692444a974d3990ff77fd1646408d"
I20260807 15:09:33.469939 18466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5c004fc3e7f47bd9001b84a954b2a73" candidate_uuid: "81140df0b6cf4beb95c4bccc5b36966b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da"
I20260807 15:09:33.470026 18257 raft_consensus.cc:3037] T b5c004fc3e7f47bd9001b84a954b2a73 P d46692444a974d3990ff77fd1646408d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:33.470093 18466 raft_consensus.cc:3037] T b5c004fc3e7f47bd9001b84a954b2a73 P e16f82b89e3a4e0ab2ccdba725cd36da [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:33.471522 18257 raft_consensus.cc:2454] T b5c004fc3e7f47bd9001b84a954b2a73 P d46692444a974d3990ff77fd1646408d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81140df0b6cf4beb95c4bccc5b36966b in term 1.
I20260807 15:09:33.471915 17968 leader_election.cc:302] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81140df0b6cf4beb95c4bccc5b36966b, d46692444a974d3990ff77fd1646408d; no voters:
I20260807 15:09:33.472108 18466 raft_consensus.cc:2454] T b5c004fc3e7f47bd9001b84a954b2a73 P e16f82b89e3a4e0ab2ccdba725cd36da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81140df0b6cf4beb95c4bccc5b36966b in term 1.
I20260807 15:09:33.472119 18576 raft_consensus.cc:2781] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:33.472389 18576 raft_consensus.cc:686] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b [term 1 LEADER]: Becoming Leader. State: Replica: 81140df0b6cf4beb95c4bccc5b36966b, State: Running, Role: LEADER
I20260807 15:09:33.472792 18576 consensus_queue.cc:237] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.476142 17833 catalog_manager.cc:5495] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b reported cstate change: term changed from 0 to 1, leader changed from <none> to 81140df0b6cf4beb95c4bccc5b36966b (127.15.226.65). New cstate: current_term: 1 leader_uuid: "81140df0b6cf4beb95c4bccc5b36966b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } health_report { overall_health: UNKNOWN } } }
I20260807 15:09:33.506740 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.502359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51006 (local address 127.15.226.65:38835)
0807 15:09:33.502528 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:33.502533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:33.502552 (+ 19us) server_negotiation.cc:407] Connection header received
0807 15:09:33.502762 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:33.502765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:33.502827 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:33.502945 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:09:33.503236 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.503765 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.504843 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.505069 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.505197 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.505596 (+ 399us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:09:33.506131 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:09:33.506329 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:33.506406 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 15:09:33.506476 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 15:09:33.514298 18466 raft_consensus.cc:1261] T b5c004fc3e7f47bd9001b84a954b2a73 P e16f82b89e3a4e0ab2ccdba725cd36da [term 1 FOLLOWER]: Refusing update from remote peer 81140df0b6cf4beb95c4bccc5b36966b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:33.515205 18576 consensus_queue.cc:1035] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:33.516424 18257 raft_consensus.cc:1261] T b5c004fc3e7f47bd9001b84a954b2a73 P d46692444a974d3990ff77fd1646408d [term 1 FOLLOWER]: Refusing update from remote peer 81140df0b6cf4beb95c4bccc5b36966b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:33.517335 18576 consensus_queue.cc:1035] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:33.525775 18586 mvcc.cc:205] Tried to move back new op lower bound from 7315928569903468544 to 7315928569747230720. Current Snapshot: MvccSnapshot[applied={T|T < 7315928569903468544}]
I20260807 15:09:33.527486 18589 mvcc.cc:205] Tried to move back new op lower bound from 7315928569903468544 to 7315928569747230720. Current Snapshot: MvccSnapshot[applied={T|T < 7315928569903468544}]
I20260807 15:09:33.540238 17834 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:33.540613 17834 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:33.543820 17834 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12: Sending DeleteTablet for 3 replicas of tablet b5c004fc3e7f47bd9001b84a954b2a73
I20260807 15:09:33.544855 18033 tablet_service.cc:1514] Processing DeleteTablet for tablet b5c004fc3e7f47bd9001b84a954b2a73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:33 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:50998
I20260807 15:09:33.545053 18433 tablet_service.cc:1514] Processing DeleteTablet for tablet b5c004fc3e7f47bd9001b84a954b2a73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:33 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:51022
I20260807 15:09:33.545028 18230 tablet_service.cc:1514] Processing DeleteTablet for tablet b5c004fc3e7f47bd9001b84a954b2a73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:33 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:48726
I20260807 15:09:33.545614 18600 tablet_replica.cc:330] T b5c004fc3e7f47bd9001b84a954b2a73 P e16f82b89e3a4e0ab2ccdba725cd36da: stopping tablet replica
I20260807 15:09:33.545645 18599 tablet_replica.cc:330] T b5c004fc3e7f47bd9001b84a954b2a73 P d46692444a974d3990ff77fd1646408d: stopping tablet replica
I20260807 15:09:33.545925 18600 raft_consensus.cc:2229] T b5c004fc3e7f47bd9001b84a954b2a73 P e16f82b89e3a4e0ab2ccdba725cd36da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:33.546007 18599 raft_consensus.cc:2229] T b5c004fc3e7f47bd9001b84a954b2a73 P d46692444a974d3990ff77fd1646408d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:33.546227 18600 raft_consensus.cc:2258] T b5c004fc3e7f47bd9001b84a954b2a73 P e16f82b89e3a4e0ab2ccdba725cd36da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:33.546392 18599 raft_consensus.cc:2258] T b5c004fc3e7f47bd9001b84a954b2a73 P d46692444a974d3990ff77fd1646408d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:33.548164 18599 ts_tablet_manager.cc:1905] T b5c004fc3e7f47bd9001b84a954b2a73 P d46692444a974d3990ff77fd1646408d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:33.548327 18600 ts_tablet_manager.cc:1905] T b5c004fc3e7f47bd9001b84a954b2a73 P e16f82b89e3a4e0ab2ccdba725cd36da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:33.548661 17834 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55464:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 15:09:33.549067 18602 tablet_replica.cc:330] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b: stopping tablet replica
W20260807 15:09:33.549203 17834 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:09:33.549575 18602 raft_consensus.cc:2229] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:09:33.550066 18602 raft_consensus.cc:2258] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:33.551592 18599 ts_tablet_manager.cc:1918] T b5c004fc3e7f47bd9001b84a954b2a73 P d46692444a974d3990ff77fd1646408d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:09:33.551731 18599 log.cc:1196] T b5c004fc3e7f47bd9001b84a954b2a73 P d46692444a974d3990ff77fd1646408d: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/b5c004fc3e7f47bd9001b84a954b2a73
I20260807 15:09:33.551940 18602 ts_tablet_manager.cc:1905] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:33.552099 18599 ts_tablet_manager.cc:1939] T b5c004fc3e7f47bd9001b84a954b2a73 P d46692444a974d3990ff77fd1646408d: Deleting consensus metadata
I20260807 15:09:33.553586 17819 catalog_manager.cc:4841] TS d46692444a974d3990ff77fd1646408d (127.15.226.66:44589): tablet b5c004fc3e7f47bd9001b84a954b2a73 (table test-table [id=2bf252edcace414abfc71d42f8904716]) successfully deleted
I20260807 15:09:33.556218 18602 ts_tablet_manager.cc:1918] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:09:33.556506 18602 log.cc:1196] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-0/wal/wals/b5c004fc3e7f47bd9001b84a954b2a73
I20260807 15:09:33.556990 18602 ts_tablet_manager.cc:1939] T b5c004fc3e7f47bd9001b84a954b2a73 P 81140df0b6cf4beb95c4bccc5b36966b: Deleting consensus metadata
I20260807 15:09:33.557049 18433 tablet_service.cc:1467] Processing CreateTablet for tablet e0d7f0e5afd941f8838d47d788261ba5 (DEFAULT_TABLE table=test-table [id=47b7685213f24f84b3b06b0817dfdcdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:33.557370 18433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0d7f0e5afd941f8838d47d788261ba5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:33.559477 18230 tablet_service.cc:1467] Processing CreateTablet for tablet e0d7f0e5afd941f8838d47d788261ba5 (DEFAULT_TABLE table=test-table [id=47b7685213f24f84b3b06b0817dfdcdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:33.559660 18033 tablet_service.cc:1467] Processing CreateTablet for tablet e0d7f0e5afd941f8838d47d788261ba5 (DEFAULT_TABLE table=test-table [id=47b7685213f24f84b3b06b0817dfdcdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:33.559826 18230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0d7f0e5afd941f8838d47d788261ba5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:33.559903 18033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0d7f0e5afd941f8838d47d788261ba5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:33.559955 18566 tablet_bootstrap.cc:492] T e0d7f0e5afd941f8838d47d788261ba5 P e16f82b89e3a4e0ab2ccdba725cd36da: Bootstrap starting.
I20260807 15:09:33.561364 18566 tablet_bootstrap.cc:654] T e0d7f0e5afd941f8838d47d788261ba5 P e16f82b89e3a4e0ab2ccdba725cd36da: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:33.562906 18566 tablet_bootstrap.cc:492] T e0d7f0e5afd941f8838d47d788261ba5 P e16f82b89e3a4e0ab2ccdba725cd36da: No bootstrap required, opened a new log
I20260807 15:09:33.562897 17819 catalog_manager.cc:4841] TS 81140df0b6cf4beb95c4bccc5b36966b (127.15.226.65:38835): tablet b5c004fc3e7f47bd9001b84a954b2a73 (table test-table [id=2bf252edcace414abfc71d42f8904716]) successfully deleted
I20260807 15:09:33.563020 18566 ts_tablet_manager.cc:1397] T e0d7f0e5afd941f8838d47d788261ba5 P e16f82b89e3a4e0ab2ccdba725cd36da: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:09:33.563082 18564 tablet_bootstrap.cc:492] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b: Bootstrap starting.
I20260807 15:09:33.563419 18563 tablet_bootstrap.cc:492] T e0d7f0e5afd941f8838d47d788261ba5 P d46692444a974d3990ff77fd1646408d: Bootstrap starting.
I20260807 15:09:33.563632 18566 raft_consensus.cc:348] T e0d7f0e5afd941f8838d47d788261ba5 P e16f82b89e3a4e0ab2ccdba725cd36da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.563771 18566 raft_consensus.cc:374] T e0d7f0e5afd941f8838d47d788261ba5 P e16f82b89e3a4e0ab2ccdba725cd36da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:33.563805 18566 raft_consensus.cc:729] T e0d7f0e5afd941f8838d47d788261ba5 P e16f82b89e3a4e0ab2ccdba725cd36da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e16f82b89e3a4e0ab2ccdba725cd36da, State: Initialized, Role: FOLLOWER
I20260807 15:09:33.563935 18566 consensus_queue.cc:260] T e0d7f0e5afd941f8838d47d788261ba5 P e16f82b89e3a4e0ab2ccdba725cd36da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.564260 18566 ts_tablet_manager.cc:1428] T e0d7f0e5afd941f8838d47d788261ba5 P e16f82b89e3a4e0ab2ccdba725cd36da: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 15:09:33.565030 18563 tablet_bootstrap.cc:654] T e0d7f0e5afd941f8838d47d788261ba5 P d46692444a974d3990ff77fd1646408d: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:33.565074 18564 tablet_bootstrap.cc:654] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:33.567416 18564 tablet_bootstrap.cc:492] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b: No bootstrap required, opened a new log
I20260807 15:09:33.567610 18564 ts_tablet_manager.cc:1397] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260807 15:09:33.568207 18564 raft_consensus.cc:348] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.568363 18564 raft_consensus.cc:374] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:33.568409 18564 raft_consensus.cc:729] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81140df0b6cf4beb95c4bccc5b36966b, State: Initialized, Role: FOLLOWER
I20260807 15:09:33.568616 18564 consensus_queue.cc:260] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.569031 18564 ts_tablet_manager.cc:1428] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:09:33.569924 18600 ts_tablet_manager.cc:1918] T b5c004fc3e7f47bd9001b84a954b2a73 P e16f82b89e3a4e0ab2ccdba725cd36da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:09:33.570065 18600 log.cc:1196] T b5c004fc3e7f47bd9001b84a954b2a73 P e16f82b89e3a4e0ab2ccdba725cd36da: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/b5c004fc3e7f47bd9001b84a954b2a73
I20260807 15:09:33.570500 18600 ts_tablet_manager.cc:1939] T b5c004fc3e7f47bd9001b84a954b2a73 P e16f82b89e3a4e0ab2ccdba725cd36da: Deleting consensus metadata
I20260807 15:09:33.572240 17819 catalog_manager.cc:4841] TS e16f82b89e3a4e0ab2ccdba725cd36da (127.15.226.67:41249): tablet b5c004fc3e7f47bd9001b84a954b2a73 (table test-table [id=2bf252edcace414abfc71d42f8904716]) successfully deleted
I20260807 15:09:33.573092 18563 tablet_bootstrap.cc:492] T e0d7f0e5afd941f8838d47d788261ba5 P d46692444a974d3990ff77fd1646408d: No bootstrap required, opened a new log
I20260807 15:09:33.573222 18563 ts_tablet_manager.cc:1397] T e0d7f0e5afd941f8838d47d788261ba5 P d46692444a974d3990ff77fd1646408d: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.001s
I20260807 15:09:33.574074 18563 raft_consensus.cc:348] T e0d7f0e5afd941f8838d47d788261ba5 P d46692444a974d3990ff77fd1646408d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.574216 18563 raft_consensus.cc:374] T e0d7f0e5afd941f8838d47d788261ba5 P d46692444a974d3990ff77fd1646408d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:33.574256 18563 raft_consensus.cc:729] T e0d7f0e5afd941f8838d47d788261ba5 P d46692444a974d3990ff77fd1646408d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d46692444a974d3990ff77fd1646408d, State: Initialized, Role: FOLLOWER
I20260807 15:09:33.574393 18563 consensus_queue.cc:260] T e0d7f0e5afd941f8838d47d788261ba5 P d46692444a974d3990ff77fd1646408d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.574755 18563 ts_tablet_manager.cc:1428] T e0d7f0e5afd941f8838d47d788261ba5 P d46692444a974d3990ff77fd1646408d: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 15:09:33.608904 18576 raft_consensus.cc:482] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:09:33.609098 18576 raft_consensus.cc:504] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.609553 18576 leader_election.cc:288] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d46692444a974d3990ff77fd1646408d (127.15.226.66:44589), e16f82b89e3a4e0ab2ccdba725cd36da (127.15.226.67:41249)
I20260807 15:09:33.610085 18257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0d7f0e5afd941f8838d47d788261ba5" candidate_uuid: "81140df0b6cf4beb95c4bccc5b36966b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d46692444a974d3990ff77fd1646408d" is_pre_election: true
I20260807 15:09:33.610095 18466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0d7f0e5afd941f8838d47d788261ba5" candidate_uuid: "81140df0b6cf4beb95c4bccc5b36966b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" is_pre_election: true
I20260807 15:09:33.610246 18257 raft_consensus.cc:2454] T e0d7f0e5afd941f8838d47d788261ba5 P d46692444a974d3990ff77fd1646408d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81140df0b6cf4beb95c4bccc5b36966b in term 0.
I20260807 15:09:33.610263 18466 raft_consensus.cc:2454] T e0d7f0e5afd941f8838d47d788261ba5 P e16f82b89e3a4e0ab2ccdba725cd36da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81140df0b6cf4beb95c4bccc5b36966b in term 0.
I20260807 15:09:33.610608 17968 leader_election.cc:302] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81140df0b6cf4beb95c4bccc5b36966b, e16f82b89e3a4e0ab2ccdba725cd36da; no voters:
I20260807 15:09:33.610821 18576 raft_consensus.cc:2781] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:09:33.610890 18576 raft_consensus.cc:482] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:09:33.610929 18576 raft_consensus.cc:3037] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:33.612488 18576 raft_consensus.cc:504] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.612936 18576 leader_election.cc:288] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b [CANDIDATE]: Term 1 election: Requested vote from peers d46692444a974d3990ff77fd1646408d (127.15.226.66:44589), e16f82b89e3a4e0ab2ccdba725cd36da (127.15.226.67:41249)
I20260807 15:09:33.613487 18257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0d7f0e5afd941f8838d47d788261ba5" candidate_uuid: "81140df0b6cf4beb95c4bccc5b36966b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d46692444a974d3990ff77fd1646408d"
I20260807 15:09:33.613611 18257 raft_consensus.cc:3037] T e0d7f0e5afd941f8838d47d788261ba5 P d46692444a974d3990ff77fd1646408d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:33.613633 18466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0d7f0e5afd941f8838d47d788261ba5" candidate_uuid: "81140df0b6cf4beb95c4bccc5b36966b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da"
I20260807 15:09:33.613768 18466 raft_consensus.cc:3037] T e0d7f0e5afd941f8838d47d788261ba5 P e16f82b89e3a4e0ab2ccdba725cd36da [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:33.615031 18257 raft_consensus.cc:2454] T e0d7f0e5afd941f8838d47d788261ba5 P d46692444a974d3990ff77fd1646408d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81140df0b6cf4beb95c4bccc5b36966b in term 1.
I20260807 15:09:33.615513 17968 leader_election.cc:302] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81140df0b6cf4beb95c4bccc5b36966b, d46692444a974d3990ff77fd1646408d; no voters:
I20260807 15:09:33.615677 18466 raft_consensus.cc:2454] T e0d7f0e5afd941f8838d47d788261ba5 P e16f82b89e3a4e0ab2ccdba725cd36da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81140df0b6cf4beb95c4bccc5b36966b in term 1.
I20260807 15:09:33.615779 18576 raft_consensus.cc:2781] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:33.616221 18576 raft_consensus.cc:686] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b [term 1 LEADER]: Becoming Leader. State: Replica: 81140df0b6cf4beb95c4bccc5b36966b, State: Running, Role: LEADER
I20260807 15:09:33.616377 18576 consensus_queue.cc:237] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.618428 17834 catalog_manager.cc:5495] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b reported cstate change: term changed from 0 to 1, leader changed from <none> to 81140df0b6cf4beb95c4bccc5b36966b (127.15.226.65). New cstate: current_term: 1 leader_uuid: "81140df0b6cf4beb95c4bccc5b36966b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } health_report { overall_health: UNKNOWN } } }
Aug 07 15:09:33 dist-test-slave-3kk6 krb5kdc[17775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115372, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:33.660459 18615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.648616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:37907 (local address 127.0.0.1:55470)
0807 15:09:33.649052 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:33.649071 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:33.649222 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:33.649513 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:33.649516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:33.650182 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:33.650311 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.650320 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.651306 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.651310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.652223 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.652236 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.652760 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.652764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.652910 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.653632 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:33.653648 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:33.655761 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:33.658153 (+ 2392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:33.658165 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:33.658171 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:33.658471 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:33.658977 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:33.658980 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:33.658981 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:33.659080 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:33.659753 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:33.659760 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:33.659904 (+ 144us) client_negotiation.cc:769] Sending connection context
0807 15:09:33.660053 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 15:09:33.660301 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":41,"thread_start_us":148,"threads_started":1}
I20260807 15:09:33.660677 17909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.648720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55470 (local address 127.15.226.126:37907)
0807 15:09:33.648880 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:33.648884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:33.649198 (+ 314us) server_negotiation.cc:407] Connection header received
0807 15:09:33.649326 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:33.649328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:33.649372 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:33.649456 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:33.650510 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.651161 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.652364 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.652601 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.652688 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.655942 (+ 3254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:33.655945 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:33.655948 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:33.655994 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:33.657966 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:33.658586 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:33.658590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:33.658592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:33.658718 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:33.659260 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:33.659264 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:33.659266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:33.659555 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:33.659692 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:33.660195 (+ 503us) server_negotiation.cc:299] Negotiation successful
0807 15:09:33.660408 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":70}
I20260807 15:09:33.666981 17834 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:55470:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 15:09:33.667972 17834 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:09:33.673934 18033 tablet_service.cc:1467] Processing CreateTablet for tablet f0bdff1bbbda49d78b2eac8f9ccb3626 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b431de0df4e7491a9fc9e458ffa9eeda]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:09:33.674238 18230 tablet_service.cc:1467] Processing CreateTablet for tablet f0bdff1bbbda49d78b2eac8f9ccb3626 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b431de0df4e7491a9fc9e458ffa9eeda]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:09:33.674490 18033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0bdff1bbbda49d78b2eac8f9ccb3626. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:33.674748 18230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0bdff1bbbda49d78b2eac8f9ccb3626. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:33.677194 18564 tablet_bootstrap.cc:492] T f0bdff1bbbda49d78b2eac8f9ccb3626 P 81140df0b6cf4beb95c4bccc5b36966b: Bootstrap starting.
I20260807 15:09:33.678391 18433 tablet_service.cc:1467] Processing CreateTablet for tablet f0bdff1bbbda49d78b2eac8f9ccb3626 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b431de0df4e7491a9fc9e458ffa9eeda]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:09:33.678771 18563 tablet_bootstrap.cc:492] T f0bdff1bbbda49d78b2eac8f9ccb3626 P d46692444a974d3990ff77fd1646408d: Bootstrap starting.
I20260807 15:09:33.678854 18433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0bdff1bbbda49d78b2eac8f9ccb3626. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:33.678891 18564 tablet_bootstrap.cc:654] T f0bdff1bbbda49d78b2eac8f9ccb3626 P 81140df0b6cf4beb95c4bccc5b36966b: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:33.680207 18563 tablet_bootstrap.cc:654] T f0bdff1bbbda49d78b2eac8f9ccb3626 P d46692444a974d3990ff77fd1646408d: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:33.680967 18564 tablet_bootstrap.cc:492] T f0bdff1bbbda49d78b2eac8f9ccb3626 P 81140df0b6cf4beb95c4bccc5b36966b: No bootstrap required, opened a new log
I20260807 15:09:33.681064 18564 ts_tablet_manager.cc:1397] T f0bdff1bbbda49d78b2eac8f9ccb3626 P 81140df0b6cf4beb95c4bccc5b36966b: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260807 15:09:33.681744 18564 raft_consensus.cc:348] T f0bdff1bbbda49d78b2eac8f9ccb3626 P 81140df0b6cf4beb95c4bccc5b36966b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.681869 18564 raft_consensus.cc:374] T f0bdff1bbbda49d78b2eac8f9ccb3626 P 81140df0b6cf4beb95c4bccc5b36966b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:33.681900 18564 raft_consensus.cc:729] T f0bdff1bbbda49d78b2eac8f9ccb3626 P 81140df0b6cf4beb95c4bccc5b36966b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81140df0b6cf4beb95c4bccc5b36966b, State: Initialized, Role: FOLLOWER
I20260807 15:09:33.682035 18564 consensus_queue.cc:260] T f0bdff1bbbda49d78b2eac8f9ccb3626 P 81140df0b6cf4beb95c4bccc5b36966b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.682443 18564 ts_tablet_manager.cc:1428] T f0bdff1bbbda49d78b2eac8f9ccb3626 P 81140df0b6cf4beb95c4bccc5b36966b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:09:33.682469 18566 tablet_bootstrap.cc:492] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da: Bootstrap starting.
I20260807 15:09:33.682861 18576 tablet_replica.cc:439] T f0bdff1bbbda49d78b2eac8f9ccb3626 P 81140df0b6cf4beb95c4bccc5b36966b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } } }
I20260807 15:09:33.683039 18576 tablet_replica.cc:442] T f0bdff1bbbda49d78b2eac8f9ccb3626 P 81140df0b6cf4beb95c4bccc5b36966b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:09:33.683674 18566 tablet_bootstrap.cc:654] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:33.685397 18566 tablet_bootstrap.cc:492] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da: No bootstrap required, opened a new log
I20260807 15:09:33.685519 18566 ts_tablet_manager.cc:1397] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:09:33.685570 18563 tablet_bootstrap.cc:492] T f0bdff1bbbda49d78b2eac8f9ccb3626 P d46692444a974d3990ff77fd1646408d: No bootstrap required, opened a new log
I20260807 15:09:33.685702 18563 ts_tablet_manager.cc:1397] T f0bdff1bbbda49d78b2eac8f9ccb3626 P d46692444a974d3990ff77fd1646408d: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260807 15:09:33.686281 18566 raft_consensus.cc:348] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.686347 18563 raft_consensus.cc:348] T f0bdff1bbbda49d78b2eac8f9ccb3626 P d46692444a974d3990ff77fd1646408d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.686424 18566 raft_consensus.cc:374] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:33.686457 18563 raft_consensus.cc:374] T f0bdff1bbbda49d78b2eac8f9ccb3626 P d46692444a974d3990ff77fd1646408d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:33.686460 18566 raft_consensus.cc:729] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e16f82b89e3a4e0ab2ccdba725cd36da, State: Initialized, Role: FOLLOWER
I20260807 15:09:33.686497 18563 raft_consensus.cc:729] T f0bdff1bbbda49d78b2eac8f9ccb3626 P d46692444a974d3990ff77fd1646408d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d46692444a974d3990ff77fd1646408d, State: Initialized, Role: FOLLOWER
I20260807 15:09:33.686604 18566 consensus_queue.cc:260] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.686695 18563 consensus_queue.cc:260] T f0bdff1bbbda49d78b2eac8f9ccb3626 P d46692444a974d3990ff77fd1646408d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.686996 18566 ts_tablet_manager.cc:1428] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 15:09:33.687052 18563 ts_tablet_manager.cc:1428] T f0bdff1bbbda49d78b2eac8f9ccb3626 P d46692444a974d3990ff77fd1646408d: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 15:09:33.687073 18577 tablet_replica.cc:439] T f0bdff1bbbda49d78b2eac8f9ccb3626 P d46692444a974d3990ff77fd1646408d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } } }
I20260807 15:09:33.687243 18577 tablet_replica.cc:442] T f0bdff1bbbda49d78b2eac8f9ccb3626 P d46692444a974d3990ff77fd1646408d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:09:33.687711 18575 tablet_replica.cc:439] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } } }
I20260807 15:09:33.687899 18575 tablet_replica.cc:442] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:09:33.728606 18575 raft_consensus.cc:482] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:09:33.728886 18575 raft_consensus.cc:504] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.730135 18575 leader_election.cc:288] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81140df0b6cf4beb95c4bccc5b36966b (127.15.226.65:38835), d46692444a974d3990ff77fd1646408d (127.15.226.66:44589)
I20260807 15:09:33.735817 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.730569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:35389 (local address 127.15.226.65:38835)
0807 15:09:33.730750 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:33.730755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:33.730907 (+ 152us) server_negotiation.cc:407] Connection header received
0807 15:09:33.731155 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:33.731159 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:33.731219 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:33.731294 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:33.731840 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.732647 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.734464 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.735120 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.735257 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.735312 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:33.735619 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 15:09:33.735707 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 15:09:33.736146 18558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.731046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:40469 (local address 127.15.226.66:44589)
0807 15:09:33.731176 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:33.731181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:33.731307 (+ 126us) server_negotiation.cc:407] Connection header received
0807 15:09:33.731415 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:33.731418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:33.731469 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:33.731548 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:33.731995 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.732725 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.734142 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.734800 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.734939 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.734999 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:33.735914 (+ 915us) server_negotiation.cc:299] Negotiation successful
0807 15:09:33.736013 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:09:33.736138 18622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.730896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:44589 (local address 127.15.226.67:40469)
0807 15:09:33.731195 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:33.731209 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:33.731326 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:33.731576 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:33.731579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:33.731594 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:33.731861 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.731868 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.732832 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.732836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.734006 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.734017 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.735624 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.735628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.735755 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.735781 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 15:09:33.735863 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 15:09:33.735932 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":24,"thread_start_us":65,"threads_started":1}
I20260807 15:09:33.736618 18620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.730458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:38835 (local address 127.15.226.67:35389)
0807 15:09:33.730772 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:33.730791 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:33.731005 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:33.731465 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:33.731468 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:33.731489 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:33.731639 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.731647 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.732786 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.732789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.734340 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.734354 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.735255 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.735259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.735414 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.735476 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 15:09:33.736422 (+ 946us) client_negotiation.cc:240] Negotiation successful
0807 15:09:33.736493 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":50,"thread_start_us":111,"threads_started":1}
I20260807 15:09:33.736699 18257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0bdff1bbbda49d78b2eac8f9ccb3626" candidate_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d46692444a974d3990ff77fd1646408d" is_pre_election: true
I20260807 15:09:33.736883 18257 raft_consensus.cc:2454] T f0bdff1bbbda49d78b2eac8f9ccb3626 P d46692444a974d3990ff77fd1646408d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e16f82b89e3a4e0ab2ccdba725cd36da in term 0.
I20260807 15:09:33.737197 18063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0bdff1bbbda49d78b2eac8f9ccb3626" candidate_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81140df0b6cf4beb95c4bccc5b36966b" is_pre_election: true
I20260807 15:09:33.737255 18364 leader_election.cc:302] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d46692444a974d3990ff77fd1646408d, e16f82b89e3a4e0ab2ccdba725cd36da; no voters:
I20260807 15:09:33.737442 18063 raft_consensus.cc:2454] T f0bdff1bbbda49d78b2eac8f9ccb3626 P 81140df0b6cf4beb95c4bccc5b36966b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e16f82b89e3a4e0ab2ccdba725cd36da in term 0.
I20260807 15:09:33.737609 18575 raft_consensus.cc:2781] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:09:33.737696 18575 raft_consensus.cc:482] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:09:33.737748 18575 raft_consensus.cc:3037] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:33.739018 18575 raft_consensus.cc:504] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.739385 18575 leader_election.cc:288] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da [CANDIDATE]: Term 1 election: Requested vote from peers 81140df0b6cf4beb95c4bccc5b36966b (127.15.226.65:38835), d46692444a974d3990ff77fd1646408d (127.15.226.66:44589)
I20260807 15:09:33.739768 18063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0bdff1bbbda49d78b2eac8f9ccb3626" candidate_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81140df0b6cf4beb95c4bccc5b36966b"
I20260807 15:09:33.739820 18257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0bdff1bbbda49d78b2eac8f9ccb3626" candidate_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d46692444a974d3990ff77fd1646408d"
I20260807 15:09:33.739917 18063 raft_consensus.cc:3037] T f0bdff1bbbda49d78b2eac8f9ccb3626 P 81140df0b6cf4beb95c4bccc5b36966b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:33.739979 18257 raft_consensus.cc:3037] T f0bdff1bbbda49d78b2eac8f9ccb3626 P d46692444a974d3990ff77fd1646408d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:33.740972 18257 raft_consensus.cc:2454] T f0bdff1bbbda49d78b2eac8f9ccb3626 P d46692444a974d3990ff77fd1646408d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e16f82b89e3a4e0ab2ccdba725cd36da in term 1.
I20260807 15:09:33.741076 18063 raft_consensus.cc:2454] T f0bdff1bbbda49d78b2eac8f9ccb3626 P 81140df0b6cf4beb95c4bccc5b36966b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e16f82b89e3a4e0ab2ccdba725cd36da in term 1.
I20260807 15:09:33.741317 18364 leader_election.cc:302] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d46692444a974d3990ff77fd1646408d, e16f82b89e3a4e0ab2ccdba725cd36da; no voters:
I20260807 15:09:33.741526 18575 raft_consensus.cc:2781] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:33.741747 18575 raft_consensus.cc:686] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da [term 1 LEADER]: Becoming Leader. State: Replica: e16f82b89e3a4e0ab2ccdba725cd36da, State: Running, Role: LEADER
I20260807 15:09:33.742026 18575 consensus_queue.cc:237] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } }
I20260807 15:09:33.743461 18623 tablet_replica.cc:439] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da: TxnStatusTablet state changed. Reason: New leader e16f82b89e3a4e0ab2ccdba725cd36da. Latest consensus state: current_term: 1 leader_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } } }
I20260807 15:09:33.743590 18623 tablet_replica.cc:442] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:09:33.743937 18626 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 15:09:33.744143 18626 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 15:09:33.745011 17834 catalog_manager.cc:5495] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da reported cstate change: term changed from 0 to 1, leader changed from <none> to e16f82b89e3a4e0ab2ccdba725cd36da (127.15.226.67). New cstate: current_term: 1 leader_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } health_report { overall_health: HEALTHY } } }
I20260807 15:09:33.759616 18615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.754828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:41249 (local address 127.0.0.1:51038)
0807 15:09:33.755016 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:33.755037 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:33.755171 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:33.755619 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:33.755622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:33.755629 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:09:33.755773 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.755779 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.756900 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.756902 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.757538 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.757547 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.758185 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.758189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.758294 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.758345 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:09:33.759286 (+ 941us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:09:33.759304 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 15:09:33.759405 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 15:09:33.759497 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 15:09:33.759629 18561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.755054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51038 (local address 127.15.226.67:41249)
0807 15:09:33.755214 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:33.755219 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:33.755236 (+ 17us) server_negotiation.cc:407] Connection header received
0807 15:09:33.755319 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:33.755322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:33.755437 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:33.755549 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:09:33.755897 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.756763 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.757664 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.758020 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.758164 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.758499 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:09:33.759173 (+ 674us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:09:33.759295 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:33.759429 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 15:09:33.759490 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 15:09:33.764019 18257 raft_consensus.cc:1261] T f0bdff1bbbda49d78b2eac8f9ccb3626 P d46692444a974d3990ff77fd1646408d [term 1 FOLLOWER]: Refusing update from remote peer e16f82b89e3a4e0ab2ccdba725cd36da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:33.764179 18063 raft_consensus.cc:1261] T f0bdff1bbbda49d78b2eac8f9ccb3626 P 81140df0b6cf4beb95c4bccc5b36966b [term 1 FOLLOWER]: Refusing update from remote peer e16f82b89e3a4e0ab2ccdba725cd36da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:33.764611 18623 consensus_queue.cc:1035] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da [LEADER]: Connected to new peer: Peer: permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:33.764809 18575 consensus_queue.cc:1035] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da [LEADER]: Connected to new peer: Peer: permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:33.766497 18577 tablet_replica.cc:439] T f0bdff1bbbda49d78b2eac8f9ccb3626 P d46692444a974d3990ff77fd1646408d: TxnStatusTablet state changed. Reason: New leader e16f82b89e3a4e0ab2ccdba725cd36da. Latest consensus state: current_term: 1 leader_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } } }
I20260807 15:09:33.766628 18577 tablet_replica.cc:442] T f0bdff1bbbda49d78b2eac8f9ccb3626 P d46692444a974d3990ff77fd1646408d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:09:33.767365 18576 tablet_replica.cc:439] T f0bdff1bbbda49d78b2eac8f9ccb3626 P 81140df0b6cf4beb95c4bccc5b36966b: TxnStatusTablet state changed. Reason: New leader e16f82b89e3a4e0ab2ccdba725cd36da. Latest consensus state: current_term: 1 leader_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } } }
I20260807 15:09:33.767515 18576 tablet_replica.cc:442] T f0bdff1bbbda49d78b2eac8f9ccb3626 P 81140df0b6cf4beb95c4bccc5b36966b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:09:33.770061 18623 tablet_replica.cc:439] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } } }
I20260807 15:09:33.770206 18623 tablet_replica.cc:442] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:09:33.770812 18576 tablet_replica.cc:439] T f0bdff1bbbda49d78b2eac8f9ccb3626 P 81140df0b6cf4beb95c4bccc5b36966b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } } }
I20260807 15:09:33.770928 18576 tablet_replica.cc:442] T f0bdff1bbbda49d78b2eac8f9ccb3626 P 81140df0b6cf4beb95c4bccc5b36966b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:09:33.770960 18585 mvcc.cc:205] Tried to move back new op lower bound from 7315928570930675712 to 7315928570850414592. Current Snapshot: MvccSnapshot[applied={T|T < 7315928570930675712 or (T in {7315928570930675712})}]
I20260807 15:09:33.771020 18577 tablet_replica.cc:439] T f0bdff1bbbda49d78b2eac8f9ccb3626 P d46692444a974d3990ff77fd1646408d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } } }
I20260807 15:09:33.771137 18577 tablet_replica.cc:442] T f0bdff1bbbda49d78b2eac8f9ccb3626 P d46692444a974d3990ff77fd1646408d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:09:33.770511 18575 tablet_replica.cc:439] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81140df0b6cf4beb95c4bccc5b36966b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38835 } } peers { permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 } } peers { permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 } } }
I20260807 15:09:33.771574 18575 tablet_replica.cc:442] T f0bdff1bbbda49d78b2eac8f9ccb3626 P e16f82b89e3a4e0ab2ccdba725cd36da: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:09:33.788470 18638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.784089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:41249 (local address 127.15.226.65:35721)
0807 15:09:33.784602 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:33.784617 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:33.784720 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:33.785028 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:33.785031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:33.785038 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:09:33.785139 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.785143 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.786262 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.786266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.787001 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.787012 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.787601 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.787604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.787720 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.787782 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:09:33.788144 (+ 362us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:09:33.788152 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 15:09:33.788240 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 15:09:33.788326 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":170,"threads_started":1}
I20260807 15:09:33.788475 18561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.784262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:35721 (local address 127.15.226.67:41249)
0807 15:09:33.784436 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:33.784441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:33.784700 (+ 259us) server_negotiation.cc:407] Connection header received
0807 15:09:33.784827 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:33.784831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:33.784917 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:33.785017 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:09:33.785276 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.786100 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.787131 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.787485 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.787593 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.787938 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:09:33.788028 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:09:33.788106 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:33.788272 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 15:09:33.788330 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 15:09:33.796886 18466 raft_consensus.cc:1261] T e0d7f0e5afd941f8838d47d788261ba5 P e16f82b89e3a4e0ab2ccdba725cd36da [term 1 FOLLOWER]: Refusing update from remote peer 81140df0b6cf4beb95c4bccc5b36966b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:33.796885 18257 raft_consensus.cc:1261] T e0d7f0e5afd941f8838d47d788261ba5 P d46692444a974d3990ff77fd1646408d [term 1 FOLLOWER]: Refusing update from remote peer 81140df0b6cf4beb95c4bccc5b36966b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:33.797394 18576 consensus_queue.cc:1035] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e16f82b89e3a4e0ab2ccdba725cd36da" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 41249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:33.797649 18581 consensus_queue.cc:1035] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d46692444a974d3990ff77fd1646408d" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:33.825003 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.820990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:38025 (local address 127.15.226.65:38835)
0807 15:09:33.821139 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:33.821145 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:33.821160 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:09:33.821216 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:33.821223 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:33.821264 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:33.821355 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:09:33.821663 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.822202 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.823468 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.823831 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.823946 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.824431 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:09:33.824524 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:09:33.824606 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:33.824775 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 15:09:33.824842 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 15:09:33.825048 18645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.820521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:38835 (local address 127.15.226.67:38025)
0807 15:09:33.820955 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:33.820974 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:33.821085 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:33.821370 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:33.821373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:33.821381 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:09:33.821530 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.821536 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.822334 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.822338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.823337 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:33.823347 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.824043 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.824047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:33.824188 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.824282 (+ 94us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:09:33.824649 (+ 367us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:09:33.824656 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 15:09:33.824754 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 15:09:33.824830 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":138,"threads_started":1}
I20260807 15:09:33.853850 17833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:33.854049 17833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55464:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:33.856669 17833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 58e2a93dcda74f96b2ea13ce691c6d12: Sending DeleteTablet for 3 replicas of tablet e0d7f0e5afd941f8838d47d788261ba5
I20260807 15:09:33.857548 18230 tablet_service.cc:1514] Processing DeleteTablet for tablet e0d7f0e5afd941f8838d47d788261ba5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:33 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:48726
I20260807 15:09:33.857781 18599 tablet_replica.cc:330] T e0d7f0e5afd941f8838d47d788261ba5 P d46692444a974d3990ff77fd1646408d: stopping tablet replica
I20260807 15:09:33.857932 18599 raft_consensus.cc:2229] T e0d7f0e5afd941f8838d47d788261ba5 P d46692444a974d3990ff77fd1646408d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:33.858098 18599 raft_consensus.cc:2258] T e0d7f0e5afd941f8838d47d788261ba5 P d46692444a974d3990ff77fd1646408d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:33.858397 18033 tablet_service.cc:1514] Processing DeleteTablet for tablet e0d7f0e5afd941f8838d47d788261ba5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:33 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:50998
I20260807 15:09:33.858613 18602 tablet_replica.cc:330] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b: stopping tablet replica
I20260807 15:09:33.858633 18433 tablet_service.cc:1514] Processing DeleteTablet for tablet e0d7f0e5afd941f8838d47d788261ba5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:33 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:51022
I20260807 15:09:33.858752 18602 raft_consensus.cc:2229] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:09:33.858989 18602 raft_consensus.cc:2258] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:33.859200 18600 tablet_replica.cc:330] T e0d7f0e5afd941f8838d47d788261ba5 P e16f82b89e3a4e0ab2ccdba725cd36da: stopping tablet replica
I20260807 15:09:33.859186 18599 ts_tablet_manager.cc:1905] T e0d7f0e5afd941f8838d47d788261ba5 P d46692444a974d3990ff77fd1646408d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:33.859338 18600 raft_consensus.cc:2229] T e0d7f0e5afd941f8838d47d788261ba5 P e16f82b89e3a4e0ab2ccdba725cd36da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:33.859488 18600 raft_consensus.cc:2258] T e0d7f0e5afd941f8838d47d788261ba5 P e16f82b89e3a4e0ab2ccdba725cd36da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:33.859905 18602 ts_tablet_manager.cc:1905] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:33.861690 18600 ts_tablet_manager.cc:1905] T e0d7f0e5afd941f8838d47d788261ba5 P e16f82b89e3a4e0ab2ccdba725cd36da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:33.862718 18599 ts_tablet_manager.cc:1918] T e0d7f0e5afd941f8838d47d788261ba5 P d46692444a974d3990ff77fd1646408d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:09:33.862807 18599 log.cc:1196] T e0d7f0e5afd941f8838d47d788261ba5 P d46692444a974d3990ff77fd1646408d: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/e0d7f0e5afd941f8838d47d788261ba5
I20260807 15:09:33.863077 18599 ts_tablet_manager.cc:1939] T e0d7f0e5afd941f8838d47d788261ba5 P d46692444a974d3990ff77fd1646408d: Deleting consensus metadata
I20260807 15:09:33.863221 18602 ts_tablet_manager.cc:1918] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:09:33.863306 18602 log.cc:1196] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-0/wal/wals/e0d7f0e5afd941f8838d47d788261ba5
I20260807 15:09:33.863607 18602 ts_tablet_manager.cc:1939] T e0d7f0e5afd941f8838d47d788261ba5 P 81140df0b6cf4beb95c4bccc5b36966b: Deleting consensus metadata
I20260807 15:09:33.864524 17819 catalog_manager.cc:4841] TS d46692444a974d3990ff77fd1646408d (127.15.226.66:44589): tablet e0d7f0e5afd941f8838d47d788261ba5 (table test-table [id=47b7685213f24f84b3b06b0817dfdcdd]) successfully deleted
I20260807 15:09:33.864712 17819 catalog_manager.cc:4841] TS 81140df0b6cf4beb95c4bccc5b36966b (127.15.226.65:38835): tablet e0d7f0e5afd941f8838d47d788261ba5 (table test-table [id=47b7685213f24f84b3b06b0817dfdcdd]) successfully deleted
I20260807 15:09:33.870388 18600 ts_tablet_manager.cc:1918] T e0d7f0e5afd941f8838d47d788261ba5 P e16f82b89e3a4e0ab2ccdba725cd36da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:09:33.870513 18600 log.cc:1196] T e0d7f0e5afd941f8838d47d788261ba5 P e16f82b89e3a4e0ab2ccdba725cd36da: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/e0d7f0e5afd941f8838d47d788261ba5
I20260807 15:09:33.870875 18600 ts_tablet_manager.cc:1939] T e0d7f0e5afd941f8838d47d788261ba5 P e16f82b89e3a4e0ab2ccdba725cd36da: Deleting consensus metadata
I20260807 15:09:33.871994 17819 catalog_manager.cc:4841] TS e16f82b89e3a4e0ab2ccdba725cd36da (127.15.226.67:41249): tablet e0d7f0e5afd941f8838d47d788261ba5 (table test-table [id=47b7685213f24f84b3b06b0817dfdcdd]) successfully deleted
Aug 07 15:09:33 dist-test-slave-3kk6 krb5kdc[17775](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115373, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.226.65@KRBTEST.COM
I20260807 15:09:33.877398 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.863289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51020 (local address 127.15.226.65:38835)
0807 15:09:33.863685 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:33.863691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:33.864837 (+ 1146us) server_negotiation.cc:407] Connection header received
0807 15:09:33.864989 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:33.864993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:33.865050 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:33.865144 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:33.865887 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.866406 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.867686 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.868023 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.868112 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.873017 (+ 4905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:33.873044 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:33.873059 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:33.873091 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:33.875155 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:33.875816 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:33.875824 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:33.875826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:33.875932 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:33.876252 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:33.876255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:33.876257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:33.876558 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:33.876799 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:33.877027 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 15:09:33.877167 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308}
W20260807 15:09:33.878006 18121 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51020
I20260807 15:09:33.888614 17909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:33.880011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55478 (local address 127.15.226.126:37907)
0807 15:09:33.880153 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:33.880158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:33.880560 (+ 402us) server_negotiation.cc:407] Connection header received
0807 15:09:33.880692 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:33.880695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:33.880752 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:33.880850 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:33.881603 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.882155 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.883059 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:33.883293 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:33.883410 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:33.885020 (+ 1610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:33.885024 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:33.885027 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:33.885059 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:33.886763 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:33.887259 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:33.887263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:33.887265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:33.887363 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:33.887656 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:33.887659 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:33.887661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:33.887893 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:33.888041 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:33.888300 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 15:09:33.888438 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260807 15:09:33.889102 17833 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55478
I20260807 15:09:33.890583 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 17941
I20260807 15:09:33.905229 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 18143
I20260807 15:09:33.918955 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 18338
I20260807 15:09:33.938206 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 17793
2026-08-07T15:09:33Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3283 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:09:34 dist-test-slave-3kk6 krb5kdc[18658](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:09:34 dist-test-slave-3kk6 krb5kdc[18658](info): set up 2 sockets
Aug 07 15:09:34 dist-test-slave-3kk6 krb5kdc[18658](info): commencing operation
krb5kdc: starting...
W20260807 15:09:35.198009 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:09:35 dist-test-slave-3kk6 krb5kdc[18658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:09:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:09:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:09:35.334043 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:35401
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:34597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:35401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:35.450278 18673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:35.450568 18673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:35.450610 18673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:35.454754 18673 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:35.454834 18673 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 15:09:35.454852 18673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:35.454871 18673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:35.454887 18673 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:35.454903 18673 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:09:35.460467 18673 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:34597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:35401
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:35401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:35.461760 18673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:35.462909 18673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:35.471884 18685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:35.471972 18681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:35.471983 18682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:35.472352 18673 server_base.cc:1034] running on GCE node
I20260807 15:09:35.473012 18673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:35.474198 18673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:35.475397 18673 hybrid_clock.cc:648] HybridClock initialized: now 1786115375475367 us; error 49 us; skew 500 ppm
Aug 07 15:09:35 dist-test-slave-3kk6 krb5kdc[18658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115375, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:35.478958 18673 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:09:35.479951 18673 webserver.cc:460] Webserver started at http://127.15.226.126:41729/ using document root <none> and password file <none>
I20260807 15:09:35.480583 18673 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:35.480681 18673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:35.480935 18673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:35.483156 18673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "fc5e3fa914b44a32ba90256378122591"
format_stamp: "Formatted at 2026-08-07 15:09:35 on dist-test-slave-3kk6"
server_key: "d16f4c386486f7423222c0c8a9724264"
server_key_iv: "f5f2bda9a103ac1eea7bf861f498613d"
server_key_version: "encryptionkey@0"
I20260807 15:09:35.483790 18673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "fc5e3fa914b44a32ba90256378122591"
format_stamp: "Formatted at 2026-08-07 15:09:35 on dist-test-slave-3kk6"
server_key: "d16f4c386486f7423222c0c8a9724264"
server_key_iv: "f5f2bda9a103ac1eea7bf861f498613d"
server_key_version: "encryptionkey@0"
I20260807 15:09:35.487824 18673 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:09:35.490346 18695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:35.491618 18673 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:09:35.491825 18673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "fc5e3fa914b44a32ba90256378122591"
format_stamp: "Formatted at 2026-08-07 15:09:35 on dist-test-slave-3kk6"
server_key: "d16f4c386486f7423222c0c8a9724264"
server_key_iv: "f5f2bda9a103ac1eea7bf861f498613d"
server_key_version: "encryptionkey@0"
I20260807 15:09:35.491958 18673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:35.512463 18673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:35.516223 18673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:35.516460 18673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:35.525382 18673 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:35401
I20260807 15:09:35.525391 18788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:35401 every 8 connection(s)
I20260807 15:09:35.526422 18673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:09:35.529276 18790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:35.532133 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 18673
I20260807 15:09:35.532249 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:09:35.532558 16265 external_mini_cluster.cc:1428] Setting key fb4566124eacdd681808eae28358684e
I20260807 15:09:35.537134 18790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591: Bootstrap starting.
I20260807 15:09:35.540091 18790 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:35.541093 18790 log.cc:824] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:35.543430 18790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591: No bootstrap required, opened a new log
Aug 07 15:09:35 dist-test-slave-3kk6 krb5kdc[18658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:35.548590 18790 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5e3fa914b44a32ba90256378122591" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 35401 } }
I20260807 15:09:35.548974 18790 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:35.549039 18790 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc5e3fa914b44a32ba90256378122591, State: Initialized, Role: FOLLOWER
I20260807 15:09:35.549585 18790 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5e3fa914b44a32ba90256378122591" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 35401 } }
I20260807 15:09:35.549777 18790 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:09:35.549849 18790 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:09:35.549947 18790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:35.550453 18793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:35.534117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49714 (local address 127.15.226.126:35401)
0807 15:09:35.534837 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:35.534859 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:35.534938 (+ 79us) server_negotiation.cc:407] Connection header received
0807 15:09:35.535907 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:35.535932 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:35.536391 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:35.536788 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:35.537646 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:35.539190 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:35.540189 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:35.540920 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:35.541098 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:35.544352 (+ 3254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:35.544361 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:35.544385 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:35.544424 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:35.546789 (+ 2365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:35.547362 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:35.547369 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:35.547376 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:35.547538 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:35.547882 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:35.547885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:35.547887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:35.548492 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:35.548740 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:35.548988 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 15:09:35.549357 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":165,"threads_started":1}
I20260807 15:09:35.551363 18790 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5e3fa914b44a32ba90256378122591" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 35401 } }
I20260807 15:09:35.551788 18790 leader_election.cc:302] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc5e3fa914b44a32ba90256378122591; no voters:
I20260807 15:09:35.552217 18790 leader_election.cc:288] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:09:35.552358 18796 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:35.552613 18796 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591 [term 1 LEADER]: Becoming Leader. State: Replica: fc5e3fa914b44a32ba90256378122591, State: Running, Role: LEADER
I20260807 15:09:35.552973 18796 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5e3fa914b44a32ba90256378122591" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 35401 } }
I20260807 15:09:35.553618 18790 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:35.554364 18796 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc5e3fa914b44a32ba90256378122591. Latest consensus state: current_term: 1 leader_uuid: "fc5e3fa914b44a32ba90256378122591" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5e3fa914b44a32ba90256378122591" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 35401 } } }
I20260807 15:09:35.554509 18796 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:35.555015 18797 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc5e3fa914b44a32ba90256378122591" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc5e3fa914b44a32ba90256378122591" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 35401 } } }
I20260807 15:09:35.555207 18797 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:35.555648 18807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:09:35.559270 18807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:09:35.565912 18807 catalog_manager.cc:1347] Generated new cluster ID: f3ac8cb302b843d08e1a06254f7bd4b5
I20260807 15:09:35.566012 18807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:09:35.573516 18807 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:09:35.574587 18807 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:09:35.585402 18807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591: Generated new TSK 0
I20260807 15:09:35.586237 18807 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.226.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
I20260807 15:09:35.626096 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:35401
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:34597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:35.746747 18824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:35.747115 18824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:35.747208 18824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:35.751544 18824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:35.751665 18824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:35.751765 18824 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 15:09:35.751811 18824 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:09:35.751883 18824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:09:35.757843 18824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:34597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.226.126:35401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:35.759223 18824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:35.760331 18824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:35.770586 18835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:35.770694 18824 server_base.cc:1034] running on GCE node
W20260807 15:09:35.770470 18832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:35.770471 18833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:35.771353 18824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:35.772094 18824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:35.773286 18824 hybrid_clock.cc:648] HybridClock initialized: now 1786115375773258 us; error 46 us; skew 500 ppm
Aug 07 15:09:35 dist-test-slave-3kk6 krb5kdc[18658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115375, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:35.776352 18824 init.cc:377] Logged in from keytab as kudu/127.15.226.65@KRBTEST.COM (short username kudu)
I20260807 15:09:35.777303 18824 webserver.cc:460] Webserver started at http://127.15.226.65:34917/ using document root <none> and password file <none>
I20260807 15:09:35.777909 18824 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:35.778009 18824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:35.778261 18824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:35.780515 18824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "8b6b057ca85642cc8f73cb7dbf846a49"
format_stamp: "Formatted at 2026-08-07 15:09:35 on dist-test-slave-3kk6"
server_key: "f69650a9f04c8abb19629136063275b8"
server_key_iv: "38ebbc11fc488930ff0387dd2a7ab88d"
server_key_version: "encryptionkey@0"
I20260807 15:09:35.781132 18824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "8b6b057ca85642cc8f73cb7dbf846a49"
format_stamp: "Formatted at 2026-08-07 15:09:35 on dist-test-slave-3kk6"
server_key: "f69650a9f04c8abb19629136063275b8"
server_key_iv: "38ebbc11fc488930ff0387dd2a7ab88d"
server_key_version: "encryptionkey@0"
I20260807 15:09:35.785041 18824 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260807 15:09:35.787559 18844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:35.788841 18824 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:09:35.789002 18824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "8b6b057ca85642cc8f73cb7dbf846a49"
format_stamp: "Formatted at 2026-08-07 15:09:35 on dist-test-slave-3kk6"
server_key: "f69650a9f04c8abb19629136063275b8"
server_key_iv: "38ebbc11fc488930ff0387dd2a7ab88d"
server_key_version: "encryptionkey@0"
I20260807 15:09:35.789126 18824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:35.829245 18824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:35.833158 18824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:35.833405 18824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:35.835191 18824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:35.835268 18824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:35.835340 18824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:35.835383 18824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:09:35 dist-test-slave-3kk6 krb5kdc[18658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115375, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:35.846903 18824 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:37213
I20260807 15:09:35.847024 19013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:37213 every 8 connection(s)
I20260807 15:09:35.848146 18824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:09:35.850237 18793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:35.836118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:60067 (local address 127.15.226.126:35401)
0807 15:09:35.836290 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:35.836295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:35.837245 (+ 950us) server_negotiation.cc:407] Connection header received
0807 15:09:35.838387 (+ 1142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:35.838392 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:35.838460 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:35.838570 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:35.840091 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:35.841029 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:35.842541 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:35.842844 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:35.842963 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:35.845877 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:35.845882 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:35.845884 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:35.845918 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:35.847871 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:35.848473 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:35.848479 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:35.848484 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:35.848593 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:35.849047 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:35.849050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:35.849052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:35.849337 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:35.849447 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:35.849985 (+ 538us) server_negotiation.cc:299] Negotiation successful
0807 15:09:35.850108 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 15:09:35.851166 18857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:35.836547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:35401 (local address 127.15.226.65:60067)
0807 15:09:35.837051 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:35.837107 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:35.838108 (+ 1001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:35.838888 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:35.838896 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:35.839282 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:35.839939 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:35.839956 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:35.841175 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:35.841179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:35.842394 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:35.842404 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:35.843021 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:35.843025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:35.843217 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:35.843900 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:35.843916 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:35.845682 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:35.848004 (+ 2322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:35.848009 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:35.848021 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:35.848361 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:35.848725 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:35.848728 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:35.848731 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:35.848923 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:35.849466 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:35.849472 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:35.849716 (+ 244us) client_negotiation.cc:769] Sending connection context
0807 15:09:35.849927 (+ 211us) client_negotiation.cc:240] Negotiation successful
0807 15:09:35.850209 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":173,"threads_started":1}
I20260807 15:09:35.852581 19014 heartbeater.cc:347] Connected to a master server at 127.15.226.126:35401
I20260807 15:09:35.852737 19014 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:35.853410 19014 heartbeater.cc:511] Master 127.15.226.126:35401 requested a full tablet report, sending...
I20260807 15:09:35.855082 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 18824
I20260807 15:09:35.855221 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:09:35.855350 18720 ts_manager.cc:194] Registered new tserver with Master: 8b6b057ca85642cc8f73cb7dbf846a49 (127.15.226.65:37213)
I20260807 15:09:35.855569 16265 external_mini_cluster.cc:1428] Setting key dcbc7a83da66a0913348bb1c2c185f92
I20260807 15:09:35.856968 18720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.65@KRBTEST.COM'} at 127.15.226.65:60067
I20260807 15:09:35.864166 19021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:35.855385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:35401 (local address 127.15.226.65:48281)
0807 15:09:35.855761 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:35.855780 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:35.855891 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:35.856231 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:35.856234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:35.856490 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:35.856618 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:35.856628 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:35.857687 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:35.857690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:35.858577 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:35.858588 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:35.859472 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:35.859481 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:35.859551 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:35.860134 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:35.860146 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:35.860520 (+ 374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:35.862577 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:35.862579 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:35.862581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:35.862852 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:35.863166 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:35.863168 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:35.863170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:35.863246 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:35.863720 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:35.863722 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:35.863801 (+ 79us) client_negotiation.cc:769] Sending connection context
0807 15:09:35.863890 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 15:09:35.864005 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":122,"threads_started":1}
I20260807 15:09:35.864362 18793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:35.855816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:48281 (local address 127.15.226.126:35401)
0807 15:09:35.855936 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:35.855941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:35.855955 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:09:35.855992 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:35.855995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:35.856035 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:35.856120 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:35.856756 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:35.857479 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:35.858791 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:35.859187 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:35.859312 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:35.860673 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:35.860680 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:35.860685 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:35.860718 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:35.862461 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:35.862956 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:35.862961 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:35.862965 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:35.863064 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:35.863338 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:35.863344 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:35.863349 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:35.863620 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:35.863741 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:35.863989 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 15:09:35.864134 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
WARNING: no policy specified for kudu/127.15.226.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
I20260807 15:09:35.890007 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:35401
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:34597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:36.010453 19024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:36.010866 19024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:36.010970 19024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:36.015223 19024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:36.015290 19024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:36.015334 19024 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 15:09:36.015349 19024 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:09:36.015430 19024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:09:36.021347 19024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:34597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.226.126:35401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:36.022578 19024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:36.023725 19024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:36.033905 19034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:36.033898 19031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:36.033898 19037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:36.034140 19024 server_base.cc:1034] running on GCE node
I20260807 15:09:36.034619 19024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:36.035452 19024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:36.036651 19024 hybrid_clock.cc:648] HybridClock initialized: now 1786115376036622 us; error 41 us; skew 500 ppm
Aug 07 15:09:36 dist-test-slave-3kk6 krb5kdc[18658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115376, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:36.039742 19024 init.cc:377] Logged in from keytab as kudu/127.15.226.66@KRBTEST.COM (short username kudu)
I20260807 15:09:36.040740 19024 webserver.cc:460] Webserver started at http://127.15.226.66:33677/ using document root <none> and password file <none>
I20260807 15:09:36.041343 19024 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:36.041442 19024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:36.041683 19024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:36.043915 19024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "2b323f8a7603407b8244f42c121688a2"
format_stamp: "Formatted at 2026-08-07 15:09:36 on dist-test-slave-3kk6"
server_key: "0bc8a8d1c224ef47b24043972b7fbe19"
server_key_iv: "fcb745733d2c7047f6a30f9ca6b95d59"
server_key_version: "encryptionkey@0"
I20260807 15:09:36.044549 19024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "2b323f8a7603407b8244f42c121688a2"
format_stamp: "Formatted at 2026-08-07 15:09:36 on dist-test-slave-3kk6"
server_key: "0bc8a8d1c224ef47b24043972b7fbe19"
server_key_iv: "fcb745733d2c7047f6a30f9ca6b95d59"
server_key_version: "encryptionkey@0"
I20260807 15:09:36.048668 19024 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:09:36.051386 19046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:36.052826 19024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:09:36.052975 19024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "2b323f8a7603407b8244f42c121688a2"
format_stamp: "Formatted at 2026-08-07 15:09:36 on dist-test-slave-3kk6"
server_key: "0bc8a8d1c224ef47b24043972b7fbe19"
server_key_iv: "fcb745733d2c7047f6a30f9ca6b95d59"
server_key_version: "encryptionkey@0"
I20260807 15:09:36.053113 19024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:36.075397 19024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:36.078860 19024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:36.079090 19024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:36.080767 19024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:36.080852 19024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:36.080924 19024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:36.080971 19024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:09:36 dist-test-slave-3kk6 krb5kdc[18658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115376, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:36.093171 19024 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:42023
I20260807 15:09:36.093501 19221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:42023 every 8 connection(s)
I20260807 15:09:36.094439 19024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:09:36.096911 18793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.081689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:56431 (local address 127.15.226.126:35401)
0807 15:09:36.081856 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:36.081862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:36.082854 (+ 992us) server_negotiation.cc:407] Connection header received
0807 15:09:36.083947 (+ 1093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:36.083951 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:36.084011 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:36.084094 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:36.085875 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.086496 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.087990 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.088363 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.088505 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.092070 (+ 3565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:36.092075 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:36.092078 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:36.092110 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:36.094140 (+ 2030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:36.094805 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:36.094812 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:36.094817 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:36.094956 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:36.095407 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:36.095412 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:36.095413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:36.095698 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:36.095818 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:36.096483 (+ 665us) server_negotiation.cc:299] Negotiation successful
0807 15:09:36.096651 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 15:09:36.097591 19059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.082144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:35401 (local address 127.15.226.66:56431)
0807 15:09:36.082636 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:36.082724 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:36.083667 (+ 943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:36.084595 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:36.084608 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:36.085074 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:36.085707 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.085726 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.086639 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.086643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.087861 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.087871 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.088561 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.088565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.088761 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.089565 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:36.089582 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:36.091856 (+ 2274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:36.094266 (+ 2410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:36.094271 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:36.094283 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:36.094632 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:36.095074 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:36.095078 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:36.095080 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:36.095271 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:36.095873 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:36.095880 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:36.096208 (+ 328us) client_negotiation.cc:769] Sending connection context
0807 15:09:36.096371 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 15:09:36.096651 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":148,"threads_started":1}
I20260807 15:09:36.098924 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 19024
I20260807 15:09:36.099048 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:09:36.099412 16265 external_mini_cluster.cc:1428] Setting key 21e282fbe80ec56d986a69bd01559433
I20260807 15:09:36.100666 19222 heartbeater.cc:347] Connected to a master server at 127.15.226.126:35401
I20260807 15:09:36.101011 19222 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:36.101559 19222 heartbeater.cc:511] Master 127.15.226.126:35401 requested a full tablet report, sending...
I20260807 15:09:36.102811 18720 ts_manager.cc:194] Registered new tserver with Master: 2b323f8a7603407b8244f42c121688a2 (127.15.226.66:42023)
I20260807 15:09:36.103645 18720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.66@KRBTEST.COM'} at 127.15.226.66:56431
WARNING: no policy specified for kudu/127.15.226.67@KRBTEST.COM; defaulting to no policy
I20260807 15:09:36.114507 18793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.106851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:48085 (local address 127.15.226.126:35401)
0807 15:09:36.107029 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:36.107036 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:36.107223 (+ 187us) server_negotiation.cc:407] Connection header received
0807 15:09:36.107319 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:36.107322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:36.107390 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:36.107499 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:36.107995 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.108529 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.109413 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.109645 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.109752 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.111158 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:36.111162 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:36.111165 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:36.111197 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:36.112748 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:36.113222 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:36.113226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:36.113228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:36.113358 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:36.113636 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:36.113639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:36.113640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:36.113890 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:36.113976 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:36.114256 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 15:09:36.114381 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 15:09:36.114527 19233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.106769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:35401 (local address 127.15.226.66:48085)
0807 15:09:36.107131 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:36.107145 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:36.107240 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:36.107522 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:36.107525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:36.107757 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:36.107868 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.107875 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.108667 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.108671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.109276 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.109285 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.109771 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.109774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.109876 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.110538 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:36.110550 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:36.111014 (+ 464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:36.112874 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:36.112877 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:36.112880 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:36.113090 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:36.113467 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:36.113469 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:36.113470 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:36.113550 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:36.113982 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:36.113984 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:36.114068 (+ 84us) client_negotiation.cc:769] Sending connection context
0807 15:09:36.114150 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 15:09:36.114314 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":108,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
I20260807 15:09:36.138206 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:35401
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:34597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:36.260594 19238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:36.260982 19238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:36.261039 19238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:36.265285 19238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:36.265429 19238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:36.265475 19238 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 15:09:36.265496 19238 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:09:36.265583 19238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:09:36.271612 19238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:34597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.226.126:35401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:36.272794 19238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:36.273819 19238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:36.284224 19247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:36.284228 19253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:36.284276 19238 server_base.cc:1034] running on GCE node
W20260807 15:09:36.284228 19249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:36.284947 19238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:36.285718 19238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:36.286895 19238 hybrid_clock.cc:648] HybridClock initialized: now 1786115376286872 us; error 57 us; skew 500 ppm
Aug 07 15:09:36 dist-test-slave-3kk6 krb5kdc[18658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115376, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:36.289907 19238 init.cc:377] Logged in from keytab as kudu/127.15.226.67@KRBTEST.COM (short username kudu)
I20260807 15:09:36.290983 19238 webserver.cc:460] Webserver started at http://127.15.226.67:45315/ using document root <none> and password file <none>
I20260807 15:09:36.291622 19238 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:36.291736 19238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:36.292037 19238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:36.294422 19238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "06edd948f3f14a619abe46175f7faa00"
format_stamp: "Formatted at 2026-08-07 15:09:36 on dist-test-slave-3kk6"
server_key: "252bd00320910797fc71779533d9a825"
server_key_iv: "cce014c13d2f6c9a4b8ab19bcaf2e937"
server_key_version: "encryptionkey@0"
I20260807 15:09:36.295109 19238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "06edd948f3f14a619abe46175f7faa00"
format_stamp: "Formatted at 2026-08-07 15:09:36 on dist-test-slave-3kk6"
server_key: "252bd00320910797fc71779533d9a825"
server_key_iv: "cce014c13d2f6c9a4b8ab19bcaf2e937"
server_key_version: "encryptionkey@0"
I20260807 15:09:36.299326 19238 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:09:36.302142 19263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:36.303475 19238 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:09:36.303628 19238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "06edd948f3f14a619abe46175f7faa00"
format_stamp: "Formatted at 2026-08-07 15:09:36 on dist-test-slave-3kk6"
server_key: "252bd00320910797fc71779533d9a825"
server_key_iv: "cce014c13d2f6c9a4b8ab19bcaf2e937"
server_key_version: "encryptionkey@0"
I20260807 15:09:36.303761 19238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:36.325654 19238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:36.328836 19238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:36.329072 19238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:36.330613 19238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:36.330724 19238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:36.330799 19238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:36.330852 19238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:09:36 dist-test-slave-3kk6 krb5kdc[18658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115376, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:36.343086 19238 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:45979
I20260807 15:09:36.343154 19420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:45979 every 8 connection(s)
I20260807 15:09:36.344281 19238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:09:36.345762 18793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.331920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:58699 (local address 127.15.226.126:35401)
0807 15:09:36.332068 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:36.332071 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:36.333129 (+ 1058us) server_negotiation.cc:407] Connection header received
0807 15:09:36.334324 (+ 1195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:36.334327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:36.334390 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:36.334494 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:36.335940 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.336485 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.337900 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.338258 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.338377 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.341580 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:36.341584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:36.341587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:36.341618 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:36.343454 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:36.343993 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:36.343997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:36.344000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:36.344099 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:36.344507 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:36.344511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:36.344512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:36.344716 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:36.344844 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:36.345454 (+ 610us) server_negotiation.cc:299] Negotiation successful
0807 15:09:36.345588 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 15:09:36.346535 19274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.332404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:35401 (local address 127.15.226.67:58699)
0807 15:09:36.332951 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:36.332995 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:36.334055 (+ 1060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:36.334726 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:36.334736 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:36.335214 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:36.335786 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.335802 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.336619 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.336623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.337755 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.337765 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.338435 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.338438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.338625 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.339432 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:36.339460 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:36.341391 (+ 1931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:36.343572 (+ 2181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:36.343576 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:36.343588 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:36.343875 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:36.344199 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:36.344202 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:36.344203 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:36.344399 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:36.344897 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:36.344903 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:36.345202 (+ 299us) client_negotiation.cc:769] Sending connection context
0807 15:09:36.345392 (+ 190us) client_negotiation.cc:240] Negotiation successful
0807 15:09:36.345718 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":150,"threads_started":1}
I20260807 15:09:36.347282 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 19238
I20260807 15:09:36.347415 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:09:36.347754 16265 external_mini_cluster.cc:1428] Setting key 0f01fa290abb2dbdd65b5dbf19f3820f
I20260807 15:09:36.348054 19421 heartbeater.cc:347] Connected to a master server at 127.15.226.126:35401
I20260807 15:09:36.348209 19421 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:36.348839 19421 heartbeater.cc:511] Master 127.15.226.126:35401 requested a full tablet report, sending...
I20260807 15:09:36.350270 18719 ts_manager.cc:194] Registered new tserver with Master: 06edd948f3f14a619abe46175f7faa00 (127.15.226.67:45979)
I20260807 15:09:36.351217 18719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.67@KRBTEST.COM'} at 127.15.226.67:58699
I20260807 15:09:36.358891 18793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.350474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:46709 (local address 127.15.226.126:35401)
0807 15:09:36.350795 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:36.350798 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:36.350811 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:09:36.350923 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:36.350926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:36.350975 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:36.351042 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:36.351547 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.352297 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.353958 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.354241 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.354352 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.355630 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:36.355634 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:36.355636 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:36.355662 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:36.357048 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:36.357476 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:36.357480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:36.357482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:36.357570 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:36.357861 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:36.357865 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:36.357867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:36.358117 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:36.358233 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:36.358559 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 15:09:36.358741 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243}
I20260807 15:09:36.358901 19431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.350338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:35401 (local address 127.15.226.67:46709)
0807 15:09:36.350685 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:36.350702 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:36.350799 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:36.351081 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:36.351084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:36.351319 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:36.351445 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.351454 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.353275 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.353277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.353828 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.353836 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.354355 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.354359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.354457 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.355133 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:36.355145 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:36.355490 (+ 345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:36.357154 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:36.357159 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:36.357161 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:36.357373 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:36.357667 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:36.357670 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:36.357672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:36.357767 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:36.358214 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:36.358218 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:36.358321 (+ 103us) client_negotiation.cc:769] Sending connection context
0807 15:09:36.358432 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 15:09:36.358634 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":95,"threads_started":1}
I20260807 15:09:36.361929 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 15:09:36 dist-test-slave-3kk6 krb5kdc[18658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 15:09:36 dist-test-slave-3kk6 krb5kdc[18658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115376, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:36.389953 18793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.381300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49726 (local address 127.15.226.126:35401)
0807 15:09:36.381437 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:36.381441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:36.381453 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:09:36.381499 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:36.381501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:36.381541 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:36.381612 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:36.382392 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.382981 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.383857 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.384064 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.384142 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.386846 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:36.386849 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:36.386851 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:36.386877 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:36.388210 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:36.388649 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:36.388653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:36.388654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:36.388738 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:36.389067 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:36.389069 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:36.389070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:36.389245 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:36.389356 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:36.389654 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 15:09:36.389825 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 15:09:36.393241 18719 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:09:36.396726 18719 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:09:36.412206 19445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.405591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:45979 (local address 127.0.0.1:46340)
0807 15:09:36.406805 (+ 1214us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:36.406819 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:36.406909 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:36.407649 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:36.407651 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:36.407669 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:36.407764 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.407770 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.409017 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.409022 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.410742 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.410754 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.411566 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.411570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.411681 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.411752 (+ 71us) client_negotiation.cc:769] Sending connection context
0807 15:09:36.411893 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 15:09:36.412023 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1153,"mutex_wait_us":156,"thread_start_us":102,"threads_started":1}
I20260807 15:09:36.412190 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.405730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46340 (local address 127.15.226.67:45979)
0807 15:09:36.406361 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:36.406375 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:36.407000 (+ 625us) server_negotiation.cc:407] Connection header received
0807 15:09:36.407128 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:36.407134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:36.407313 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:36.407481 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:36.407900 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.408902 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.410896 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.411432 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.411565 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.411758 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:36.411926 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 15:09:36.412023 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":125,"threads_started":1}
I20260807 15:09:36.412452 19449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.406416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:37213 (local address 127.0.0.1:46396)
0807 15:09:36.407163 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:36.407178 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:36.407280 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:36.407979 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:36.407982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:36.408001 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:36.408109 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.408115 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.409390 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.409395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.410477 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.410487 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.411811 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.411814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.411914 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.411942 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 15:09:36.412018 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 15:09:36.412093 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":99,"threads_started":1}
I20260807 15:09:36.412622 19444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.405591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:42023 (local address 127.0.0.1:42468)
0807 15:09:36.406346 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:36.406405 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:36.406541 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:36.407192 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:36.407198 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:36.407222 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:36.407395 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.407406 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.409129 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.409132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.410742 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.410754 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.412000 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.412006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.412097 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.412117 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 15:09:36.412185 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 15:09:36.412242 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"mutex_wait_us":163,"spinlock_wait_cycles":11904,"thread_start_us":164,"threads_started":1}
I20260807 15:09:36.413134 19447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.405935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42468 (local address 127.15.226.66:42023)
0807 15:09:36.406558 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:36.406570 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:36.406599 (+ 29us) server_negotiation.cc:407] Connection header received
0807 15:09:36.406716 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:36.406722 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:36.407018 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:36.407541 (+ 523us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:36.407673 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.408998 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.410883 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.411654 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.411780 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.411938 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:36.412885 (+ 947us) server_negotiation.cc:299] Negotiation successful
0807 15:09:36.412988 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":112,"threads_started":1}
I20260807 15:09:36.413125 19450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.406587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46396 (local address 127.15.226.65:37213)
0807 15:09:36.407584 (+ 997us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:36.407590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:36.407618 (+ 28us) server_negotiation.cc:407] Connection header received
0807 15:09:36.407679 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:36.407682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:36.407812 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:36.407989 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:36.408271 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.409273 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.410605 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.411458 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.411568 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.411742 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:36.412922 (+ 1180us) server_negotiation.cc:299] Negotiation successful
0807 15:09:36.413010 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":890,"thread_start_us":137,"threads_started":1}
I20260807 15:09:36.415522 19331 tablet_service.cc:1467] Processing CreateTablet for tablet a049f60ecc1444058f0a2c310334d284 (DEFAULT_TABLE table=test-table [id=f35bd5a878d54b709f399aefff4edd92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:36.415648 18915 tablet_service.cc:1467] Processing CreateTablet for tablet a049f60ecc1444058f0a2c310334d284 (DEFAULT_TABLE table=test-table [id=f35bd5a878d54b709f399aefff4edd92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:36.415791 19119 tablet_service.cc:1467] Processing CreateTablet for tablet a049f60ecc1444058f0a2c310334d284 (DEFAULT_TABLE table=test-table [id=f35bd5a878d54b709f399aefff4edd92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:36.416657 18915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a049f60ecc1444058f0a2c310334d284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:36.416702 19331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a049f60ecc1444058f0a2c310334d284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:36.416689 19119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a049f60ecc1444058f0a2c310334d284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:36.423107 19452 tablet_bootstrap.cc:492] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49: Bootstrap starting.
I20260807 15:09:36.424592 19454 tablet_bootstrap.cc:492] T a049f60ecc1444058f0a2c310334d284 P 06edd948f3f14a619abe46175f7faa00: Bootstrap starting.
I20260807 15:09:36.425710 19452 tablet_bootstrap.cc:654] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:36.426764 19452 log.cc:824] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:36.426831 19454 tablet_bootstrap.cc:654] T a049f60ecc1444058f0a2c310334d284 P 06edd948f3f14a619abe46175f7faa00: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:36.427066 19453 tablet_bootstrap.cc:492] T a049f60ecc1444058f0a2c310334d284 P 2b323f8a7603407b8244f42c121688a2: Bootstrap starting.
I20260807 15:09:36.427659 19454 log.cc:824] T a049f60ecc1444058f0a2c310334d284 P 06edd948f3f14a619abe46175f7faa00: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:36.429049 19452 tablet_bootstrap.cc:492] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49: No bootstrap required, opened a new log
I20260807 15:09:36.429092 19453 tablet_bootstrap.cc:654] T a049f60ecc1444058f0a2c310334d284 P 2b323f8a7603407b8244f42c121688a2: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:36.429266 19452 ts_tablet_manager.cc:1397] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 15:09:36.429742 19454 tablet_bootstrap.cc:492] T a049f60ecc1444058f0a2c310334d284 P 06edd948f3f14a619abe46175f7faa00: No bootstrap required, opened a new log
I20260807 15:09:36.429903 19453 log.cc:824] T a049f60ecc1444058f0a2c310334d284 P 2b323f8a7603407b8244f42c121688a2: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:36.429942 19454 ts_tablet_manager.cc:1397] T a049f60ecc1444058f0a2c310334d284 P 06edd948f3f14a619abe46175f7faa00: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 15:09:36.431615 19453 tablet_bootstrap.cc:492] T a049f60ecc1444058f0a2c310334d284 P 2b323f8a7603407b8244f42c121688a2: No bootstrap required, opened a new log
I20260807 15:09:36.431836 19453 ts_tablet_manager.cc:1397] T a049f60ecc1444058f0a2c310334d284 P 2b323f8a7603407b8244f42c121688a2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 15:09:36.434120 19452 raft_consensus.cc:348] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } }
I20260807 15:09:36.434120 19454 raft_consensus.cc:348] T a049f60ecc1444058f0a2c310334d284 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } }
I20260807 15:09:36.434420 19452 raft_consensus.cc:374] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:36.434420 19454 raft_consensus.cc:374] T a049f60ecc1444058f0a2c310334d284 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:36.434484 19452 raft_consensus.cc:729] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b6b057ca85642cc8f73cb7dbf846a49, State: Initialized, Role: FOLLOWER
I20260807 15:09:36.434496 19454 raft_consensus.cc:729] T a049f60ecc1444058f0a2c310334d284 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06edd948f3f14a619abe46175f7faa00, State: Initialized, Role: FOLLOWER
I20260807 15:09:36.434643 19453 raft_consensus.cc:348] T a049f60ecc1444058f0a2c310334d284 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } }
I20260807 15:09:36.434928 19453 raft_consensus.cc:374] T a049f60ecc1444058f0a2c310334d284 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:36.434989 19453 raft_consensus.cc:729] T a049f60ecc1444058f0a2c310334d284 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b323f8a7603407b8244f42c121688a2, State: Initialized, Role: FOLLOWER
I20260807 15:09:36.435000 19452 consensus_queue.cc:260] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } }
I20260807 15:09:36.435000 19454 consensus_queue.cc:260] T a049f60ecc1444058f0a2c310334d284 P 06edd948f3f14a619abe46175f7faa00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } }
I20260807 15:09:36.435433 19453 consensus_queue.cc:260] T a049f60ecc1444058f0a2c310334d284 P 2b323f8a7603407b8244f42c121688a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } }
I20260807 15:09:36.436090 19452 ts_tablet_manager.cc:1428] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260807 15:09:36.436090 19454 ts_tablet_manager.cc:1428] T a049f60ecc1444058f0a2c310334d284 P 06edd948f3f14a619abe46175f7faa00: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 15:09:36.436474 19014 heartbeater.cc:503] Master 127.15.226.126:35401 was elected leader, sending a full tablet report...
I20260807 15:09:36.436474 19222 heartbeater.cc:503] Master 127.15.226.126:35401 was elected leader, sending a full tablet report...
I20260807 15:09:36.437028 19421 heartbeater.cc:503] Master 127.15.226.126:35401 was elected leader, sending a full tablet report...
I20260807 15:09:36.437166 19453 ts_tablet_manager.cc:1428] T a049f60ecc1444058f0a2c310334d284 P 2b323f8a7603407b8244f42c121688a2: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260807 15:09:36.447218 19458 raft_consensus.cc:482] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:09:36.447446 19458 raft_consensus.cc:504] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } }
I20260807 15:09:36.448752 19458 leader_election.cc:288] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06edd948f3f14a619abe46175f7faa00 (127.15.226.67:45979), 2b323f8a7603407b8244f42c121688a2 (127.15.226.66:42023)
I20260807 15:09:36.454591 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.449065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:55949 (local address 127.15.226.67:45979)
0807 15:09:36.449265 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:36.449270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:36.449620 (+ 350us) server_negotiation.cc:407] Connection header received
0807 15:09:36.449781 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:36.449783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:36.449856 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:36.449946 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:36.450370 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.451211 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.453335 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.453944 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.454073 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.454128 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:36.454421 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 15:09:36.454486 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 15:09:36.454636 19466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.448955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:45979 (local address 127.15.226.65:55949)
0807 15:09:36.449506 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:36.449523 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:36.449671 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:36.449965 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:36.449985 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:36.450010 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:36.450232 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.450240 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.451348 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.451352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.453079 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.453093 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.454142 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.454145 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.454262 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.454301 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 15:09:36.454377 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 15:09:36.454460 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":267,"spinlock_wait_cycles":22144,"thread_start_us":175,"threads_started":1}
I20260807 15:09:36.455323 19447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.449264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:60443 (local address 127.15.226.66:42023)
0807 15:09:36.449422 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:36.449425 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:36.449651 (+ 226us) server_negotiation.cc:407] Connection header received
0807 15:09:36.449862 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:36.449865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:36.449921 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:36.449999 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:36.450360 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.451207 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.453301 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.453946 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.454073 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.454128 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:36.455114 (+ 986us) server_negotiation.cc:299] Negotiation successful
0807 15:09:36.455197 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 15:09:36.455351 19465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.449061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:42023 (local address 127.15.226.65:60443)
0807 15:09:36.449527 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:36.449542 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:36.449671 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:36.450013 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:36.450015 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:36.450027 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:36.450232 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.450240 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.451348 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.451352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.453079 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.453093 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.454845 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.454849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.454947 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.454972 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 15:09:36.455091 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 15:09:36.455201 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":179,"spinlock_wait_cycles":9216,"thread_start_us":116,"threads_started":1}
I20260807 15:09:36.455529 19356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a049f60ecc1444058f0a2c310334d284" candidate_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06edd948f3f14a619abe46175f7faa00" is_pre_election: true
I20260807 15:09:36.455902 19356 raft_consensus.cc:2454] T a049f60ecc1444058f0a2c310334d284 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b6b057ca85642cc8f73cb7dbf846a49 in term 0.
I20260807 15:09:36.455885 19151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a049f60ecc1444058f0a2c310334d284" candidate_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b323f8a7603407b8244f42c121688a2" is_pre_election: true
I20260807 15:09:36.456187 19151 raft_consensus.cc:2454] T a049f60ecc1444058f0a2c310334d284 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b6b057ca85642cc8f73cb7dbf846a49 in term 0.
I20260807 15:09:36.456466 18848 leader_election.cc:302] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06edd948f3f14a619abe46175f7faa00, 8b6b057ca85642cc8f73cb7dbf846a49; no voters:
I20260807 15:09:36.456758 19458 raft_consensus.cc:2781] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:09:36.456862 19458 raft_consensus.cc:482] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:09:36.456910 19458 raft_consensus.cc:3037] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:36.458097 19458 raft_consensus.cc:504] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } }
I20260807 15:09:36.458519 19458 leader_election.cc:288] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49 [CANDIDATE]: Term 1 election: Requested vote from peers 06edd948f3f14a619abe46175f7faa00 (127.15.226.67:45979), 2b323f8a7603407b8244f42c121688a2 (127.15.226.66:42023)
I20260807 15:09:36.458930 19356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a049f60ecc1444058f0a2c310334d284" candidate_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06edd948f3f14a619abe46175f7faa00"
I20260807 15:09:36.458986 19151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a049f60ecc1444058f0a2c310334d284" candidate_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b323f8a7603407b8244f42c121688a2"
I20260807 15:09:36.459062 19356 raft_consensus.cc:3037] T a049f60ecc1444058f0a2c310334d284 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:36.459153 19151 raft_consensus.cc:3037] T a049f60ecc1444058f0a2c310334d284 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:36.460276 19356 raft_consensus.cc:2454] T a049f60ecc1444058f0a2c310334d284 P 06edd948f3f14a619abe46175f7faa00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b6b057ca85642cc8f73cb7dbf846a49 in term 1.
I20260807 15:09:36.460439 19151 raft_consensus.cc:2454] T a049f60ecc1444058f0a2c310334d284 P 2b323f8a7603407b8244f42c121688a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b6b057ca85642cc8f73cb7dbf846a49 in term 1.
I20260807 15:09:36.460692 18848 leader_election.cc:302] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06edd948f3f14a619abe46175f7faa00, 8b6b057ca85642cc8f73cb7dbf846a49; no voters:
I20260807 15:09:36.460881 19458 raft_consensus.cc:2781] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:36.461146 19458 raft_consensus.cc:686] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 1 LEADER]: Becoming Leader. State: Replica: 8b6b057ca85642cc8f73cb7dbf846a49, State: Running, Role: LEADER
I20260807 15:09:36.461531 19458 consensus_queue.cc:237] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } }
I20260807 15:09:36.465063 18719 catalog_manager.cc:5495] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b6b057ca85642cc8f73cb7dbf846a49 (127.15.226.65). New cstate: current_term: 1 leader_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } health_report { overall_health: HEALTHY } } }
Aug 07 15:09:36 dist-test-slave-3kk6 krb5kdc[18658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115375, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:36.517371 19477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.508667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:35401 (local address 127.0.0.1:49730)
0807 15:09:36.509010 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:36.509025 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:36.509136 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:36.509371 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:36.509373 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:36.509779 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:36.509879 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.509885 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.510614 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.510616 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.511261 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.511271 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.511661 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.511664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.511741 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.512319 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:36.512331 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:36.513845 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:36.515774 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:36.515783 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:36.515788 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:36.516038 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:36.516326 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:36.516328 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:36.516330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:36.516424 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:36.516833 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:36.516839 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:36.516969 (+ 130us) client_negotiation.cc:769] Sending connection context
0807 15:09:36.517073 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 15:09:36.517224 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":108,"threads_started":1}
I20260807 15:09:36.517472 18793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.508763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49730 (local address 127.15.226.126:35401)
0807 15:09:36.508959 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:36.508964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:36.509106 (+ 142us) server_negotiation.cc:407] Connection header received
0807 15:09:36.509220 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:36.509222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:36.509263 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:36.509335 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:36.510011 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.510522 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.511365 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.511567 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.511640 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.514007 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:36.514011 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:36.514013 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:36.514034 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:36.515665 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:36.516136 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:36.516138 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:36.516139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:36.516215 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:36.516526 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:36.516528 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:36.516529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:36.516704 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:36.516783 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:36.517132 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 15:09:36.517308 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":108}
I20260807 15:09:36.521507 18719 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:49730:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 15:09:36.522174 18719 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:09:36.527865 18915 tablet_service.cc:1467] Processing CreateTablet for tablet d746dd90ea924edc99887a849e1b7cd9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d1aaddae85764fe38574a58232366236]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:09:36.528136 19331 tablet_service.cc:1467] Processing CreateTablet for tablet d746dd90ea924edc99887a849e1b7cd9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d1aaddae85764fe38574a58232366236]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:09:36.528676 19331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d746dd90ea924edc99887a849e1b7cd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:36.528671 18915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d746dd90ea924edc99887a849e1b7cd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:36.531826 19452 tablet_bootstrap.cc:492] T d746dd90ea924edc99887a849e1b7cd9 P 8b6b057ca85642cc8f73cb7dbf846a49: Bootstrap starting.
I20260807 15:09:36.532398 19454 tablet_bootstrap.cc:492] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00: Bootstrap starting.
I20260807 15:09:36.532826 19452 tablet_bootstrap.cc:654] T d746dd90ea924edc99887a849e1b7cd9 P 8b6b057ca85642cc8f73cb7dbf846a49: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:36.533248 19454 tablet_bootstrap.cc:654] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:36.534474 19454 tablet_bootstrap.cc:492] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00: No bootstrap required, opened a new log
I20260807 15:09:36.534587 19454 ts_tablet_manager.cc:1397] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:09:36.534452 19119 tablet_service.cc:1467] Processing CreateTablet for tablet d746dd90ea924edc99887a849e1b7cd9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d1aaddae85764fe38574a58232366236]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:09:36.535089 19454 raft_consensus.cc:348] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:36.535197 19119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d746dd90ea924edc99887a849e1b7cd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:36.535228 19454 raft_consensus.cc:374] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:36.535272 19454 raft_consensus.cc:729] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06edd948f3f14a619abe46175f7faa00, State: Initialized, Role: FOLLOWER
I20260807 15:09:36.535449 19454 consensus_queue.cc:260] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:36.535772 19454 ts_tablet_manager.cc:1428] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 15:09:36.536010 19461 tablet_replica.cc:439] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } } }
I20260807 15:09:36.536149 19461 tablet_replica.cc:442] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:09:36.537412 19452 tablet_bootstrap.cc:492] T d746dd90ea924edc99887a849e1b7cd9 P 8b6b057ca85642cc8f73cb7dbf846a49: No bootstrap required, opened a new log
I20260807 15:09:36.537513 19452 ts_tablet_manager.cc:1397] T d746dd90ea924edc99887a849e1b7cd9 P 8b6b057ca85642cc8f73cb7dbf846a49: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260807 15:09:36.538056 19452 raft_consensus.cc:348] T d746dd90ea924edc99887a849e1b7cd9 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:36.538179 19452 raft_consensus.cc:374] T d746dd90ea924edc99887a849e1b7cd9 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:36.538187 19453 tablet_bootstrap.cc:492] T d746dd90ea924edc99887a849e1b7cd9 P 2b323f8a7603407b8244f42c121688a2: Bootstrap starting.
I20260807 15:09:36.538221 19452 raft_consensus.cc:729] T d746dd90ea924edc99887a849e1b7cd9 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b6b057ca85642cc8f73cb7dbf846a49, State: Initialized, Role: FOLLOWER
I20260807 15:09:36.538529 19452 consensus_queue.cc:260] T d746dd90ea924edc99887a849e1b7cd9 P 8b6b057ca85642cc8f73cb7dbf846a49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:36.538856 19458 tablet_replica.cc:439] T d746dd90ea924edc99887a849e1b7cd9 P 8b6b057ca85642cc8f73cb7dbf846a49: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } } }
I20260807 15:09:36.538985 19458 tablet_replica.cc:442] T d746dd90ea924edc99887a849e1b7cd9 P 8b6b057ca85642cc8f73cb7dbf846a49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:09:36.539297 19453 tablet_bootstrap.cc:654] T d746dd90ea924edc99887a849e1b7cd9 P 2b323f8a7603407b8244f42c121688a2: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:36.539762 19452 ts_tablet_manager.cc:1428] T d746dd90ea924edc99887a849e1b7cd9 P 8b6b057ca85642cc8f73cb7dbf846a49: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:09:36.540931 19453 tablet_bootstrap.cc:492] T d746dd90ea924edc99887a849e1b7cd9 P 2b323f8a7603407b8244f42c121688a2: No bootstrap required, opened a new log
I20260807 15:09:36.541036 19453 ts_tablet_manager.cc:1397] T d746dd90ea924edc99887a849e1b7cd9 P 2b323f8a7603407b8244f42c121688a2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:09:36.541501 19453 raft_consensus.cc:348] T d746dd90ea924edc99887a849e1b7cd9 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:36.541596 19453 raft_consensus.cc:374] T d746dd90ea924edc99887a849e1b7cd9 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:36.541618 19453 raft_consensus.cc:729] T d746dd90ea924edc99887a849e1b7cd9 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b323f8a7603407b8244f42c121688a2, State: Initialized, Role: FOLLOWER
I20260807 15:09:36.541776 19453 consensus_queue.cc:260] T d746dd90ea924edc99887a849e1b7cd9 P 2b323f8a7603407b8244f42c121688a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:36.542086 19453 ts_tablet_manager.cc:1428] T d746dd90ea924edc99887a849e1b7cd9 P 2b323f8a7603407b8244f42c121688a2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 15:09:36.542057 19464 tablet_replica.cc:439] T d746dd90ea924edc99887a849e1b7cd9 P 2b323f8a7603407b8244f42c121688a2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } } }
I20260807 15:09:36.542198 19464 tablet_replica.cc:442] T d746dd90ea924edc99887a849e1b7cd9 P 2b323f8a7603407b8244f42c121688a2: This TxnStatusTablet replica's current role is: FOLLOWER
W20260807 15:09:36.595469 19422 tablet.cc:2369] T a049f60ecc1444058f0a2c310334d284 P 06edd948f3f14a619abe46175f7faa00: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:09:36.600004 19015 tablet.cc:2369] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:09:36.600344 19224 tablet.cc:2369] T a049f60ecc1444058f0a2c310334d284 P 2b323f8a7603407b8244f42c121688a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:09:36.809715 19461 raft_consensus.cc:482] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:09:36.809942 19461 raft_consensus.cc:504] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:36.811286 19461 leader_election.cc:288] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b6b057ca85642cc8f73cb7dbf846a49 (127.15.226.65:37213), 2b323f8a7603407b8244f42c121688a2 (127.15.226.66:42023)
I20260807 15:09:36.816635 19447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.811883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:50499 (local address 127.15.226.66:42023)
0807 15:09:36.812028 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:36.812033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:36.812182 (+ 149us) server_negotiation.cc:407] Connection header received
0807 15:09:36.812308 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:36.812310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:36.812370 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:36.812456 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:36.812713 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.813448 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.815153 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.815811 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.815976 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.816026 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:36.816385 (+ 359us) server_negotiation.cc:299] Negotiation successful
0807 15:09:36.816496 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:09:36.816636 19274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.811526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:37213 (local address 127.15.226.67:51815)
0807 15:09:36.811673 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:36.811705 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:36.811821 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:36.812112 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:36.812116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:36.812143 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:36.812284 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.812293 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.813406 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.813410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.814982 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.814997 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.816006 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.816012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.816225 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.816242 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 15:09:36.816330 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 15:09:36.816461 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":176}
I20260807 15:09:36.816906 19450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.811669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:51815 (local address 127.15.226.65:37213)
0807 15:09:36.811839 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:36.811844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:36.811895 (+ 51us) server_negotiation.cc:407] Connection header received
0807 15:09:36.811962 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:36.811964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:36.812019 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:36.812081 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:36.812424 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.813258 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.815153 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.815811 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.815964 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.816026 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:36.816685 (+ 659us) server_negotiation.cc:299] Negotiation successful
0807 15:09:36.816782 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 15:09:36.816649 19483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.811623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:42023 (local address 127.15.226.67:50499)
0807 15:09:36.812096 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:36.812109 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:36.812200 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:36.812478 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:36.812481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:36.812496 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:36.812599 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.812603 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.813565 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.813569 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.814991 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.814997 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.816017 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.816020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.816155 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.816242 (+ 87us) client_negotiation.cc:769] Sending connection context
0807 15:09:36.816330 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 15:09:36.816461 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"mutex_wait_us":180,"thread_start_us":148,"threads_started":1}
I20260807 15:09:36.817598 18944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d746dd90ea924edc99887a849e1b7cd9" candidate_uuid: "06edd948f3f14a619abe46175f7faa00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" is_pre_election: true
I20260807 15:09:36.817642 19151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d746dd90ea924edc99887a849e1b7cd9" candidate_uuid: "06edd948f3f14a619abe46175f7faa00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b323f8a7603407b8244f42c121688a2" is_pre_election: true
I20260807 15:09:36.817833 19151 raft_consensus.cc:2454] T d746dd90ea924edc99887a849e1b7cd9 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06edd948f3f14a619abe46175f7faa00 in term 0.
I20260807 15:09:36.817860 18944 raft_consensus.cc:2454] T d746dd90ea924edc99887a849e1b7cd9 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06edd948f3f14a619abe46175f7faa00 in term 0.
I20260807 15:09:36.818467 19267 leader_election.cc:302] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 06edd948f3f14a619abe46175f7faa00, 2b323f8a7603407b8244f42c121688a2, 8b6b057ca85642cc8f73cb7dbf846a49; no voters:
I20260807 15:09:36.818863 19461 raft_consensus.cc:2781] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:09:36.818957 19461 raft_consensus.cc:482] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:09:36.819016 19461 raft_consensus.cc:3037] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:36.833599 19461 raft_consensus.cc:504] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:36.834252 19461 leader_election.cc:288] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00 [CANDIDATE]: Term 1 election: Requested vote from peers 8b6b057ca85642cc8f73cb7dbf846a49 (127.15.226.65:37213), 2b323f8a7603407b8244f42c121688a2 (127.15.226.66:42023)
I20260807 15:09:36.834774 18944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d746dd90ea924edc99887a849e1b7cd9" candidate_uuid: "06edd948f3f14a619abe46175f7faa00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b6b057ca85642cc8f73cb7dbf846a49"
I20260807 15:09:36.834832 19151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d746dd90ea924edc99887a849e1b7cd9" candidate_uuid: "06edd948f3f14a619abe46175f7faa00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b323f8a7603407b8244f42c121688a2"
I20260807 15:09:36.834942 18944 raft_consensus.cc:3037] T d746dd90ea924edc99887a849e1b7cd9 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:36.835008 19151 raft_consensus.cc:3037] T d746dd90ea924edc99887a849e1b7cd9 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:36.836098 18944 raft_consensus.cc:2454] T d746dd90ea924edc99887a849e1b7cd9 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06edd948f3f14a619abe46175f7faa00 in term 1.
I20260807 15:09:36.836539 19267 leader_election.cc:302] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06edd948f3f14a619abe46175f7faa00, 8b6b057ca85642cc8f73cb7dbf846a49; no voters:
I20260807 15:09:36.836598 19151 raft_consensus.cc:2454] T d746dd90ea924edc99887a849e1b7cd9 P 2b323f8a7603407b8244f42c121688a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06edd948f3f14a619abe46175f7faa00 in term 1.
I20260807 15:09:36.836782 19461 raft_consensus.cc:2781] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:36.837074 19461 raft_consensus.cc:686] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00 [term 1 LEADER]: Becoming Leader. State: Replica: 06edd948f3f14a619abe46175f7faa00, State: Running, Role: LEADER
I20260807 15:09:36.837592 19461 consensus_queue.cc:237] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:36.840328 19484 tablet_replica.cc:439] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00: TxnStatusTablet state changed. Reason: New leader 06edd948f3f14a619abe46175f7faa00. Latest consensus state: current_term: 1 leader_uuid: "06edd948f3f14a619abe46175f7faa00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } } }
I20260807 15:09:36.840471 19484 tablet_replica.cc:442] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:09:36.840780 19487 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 15:09:36.841038 19487 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 15:09:36.841823 18719 catalog_manager.cc:5495] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06edd948f3f14a619abe46175f7faa00 (127.15.226.67). New cstate: current_term: 1 leader_uuid: "06edd948f3f14a619abe46175f7faa00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } health_report { overall_health: HEALTHY } } }
I20260807 15:09:36.908550 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.904627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46346 (local address 127.15.226.67:45979)
0807 15:09:36.904791 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:36.904796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:36.904812 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:09:36.904998 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:36.905000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:36.905059 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:36.905167 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:09:36.905433 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.905954 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.906956 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.907175 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.907282 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.907621 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:09:36.908132 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:09:36.908259 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:36.908347 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 15:09:36.908402 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:09:36.908550 19477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.904451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:45979 (local address 127.0.0.1:46346)
0807 15:09:36.904632 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:36.904676 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:36.904804 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:36.905186 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:36.905189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:36.905197 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:09:36.905320 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.905326 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.906082 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.906086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.906806 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.906818 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.907294 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.907298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.907400 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.907456 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:09:36.908230 (+ 774us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:09:36.908248 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 15:09:36.908319 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 15:09:36.908409 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 15:09:36.913968 18944 raft_consensus.cc:1261] T d746dd90ea924edc99887a849e1b7cd9 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 1 FOLLOWER]: Refusing update from remote peer 06edd948f3f14a619abe46175f7faa00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:36.913969 19151 raft_consensus.cc:1261] T d746dd90ea924edc99887a849e1b7cd9 P 2b323f8a7603407b8244f42c121688a2 [term 1 FOLLOWER]: Refusing update from remote peer 06edd948f3f14a619abe46175f7faa00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:36.914698 19484 consensus_queue.cc:1035] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:36.914991 19461 consensus_queue.cc:1035] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:36.918578 19458 tablet_replica.cc:439] T d746dd90ea924edc99887a849e1b7cd9 P 8b6b057ca85642cc8f73cb7dbf846a49: TxnStatusTablet state changed. Reason: New leader 06edd948f3f14a619abe46175f7faa00. Latest consensus state: current_term: 1 leader_uuid: "06edd948f3f14a619abe46175f7faa00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } } }
I20260807 15:09:36.918748 19458 tablet_replica.cc:442] T d746dd90ea924edc99887a849e1b7cd9 P 8b6b057ca85642cc8f73cb7dbf846a49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:09:36.919643 19464 tablet_replica.cc:439] T d746dd90ea924edc99887a849e1b7cd9 P 2b323f8a7603407b8244f42c121688a2: TxnStatusTablet state changed. Reason: New leader 06edd948f3f14a619abe46175f7faa00. Latest consensus state: current_term: 1 leader_uuid: "06edd948f3f14a619abe46175f7faa00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } } }
I20260807 15:09:36.919776 19464 tablet_replica.cc:442] T d746dd90ea924edc99887a849e1b7cd9 P 2b323f8a7603407b8244f42c121688a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:09:36.922086 19484 tablet_replica.cc:439] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "06edd948f3f14a619abe46175f7faa00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } } }
I20260807 15:09:36.922230 19484 tablet_replica.cc:442] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:09:36.924707 19484 tablet_replica.cc:439] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "06edd948f3f14a619abe46175f7faa00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } } }
I20260807 15:09:36.924836 19484 tablet_replica.cc:442] T d746dd90ea924edc99887a849e1b7cd9 P 06edd948f3f14a619abe46175f7faa00: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:09:36.926501 19464 tablet_replica.cc:439] T d746dd90ea924edc99887a849e1b7cd9 P 2b323f8a7603407b8244f42c121688a2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "06edd948f3f14a619abe46175f7faa00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } } }
I20260807 15:09:36.926532 19458 tablet_replica.cc:439] T d746dd90ea924edc99887a849e1b7cd9 P 8b6b057ca85642cc8f73cb7dbf846a49: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "06edd948f3f14a619abe46175f7faa00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } } }
I20260807 15:09:36.926646 19464 tablet_replica.cc:442] T d746dd90ea924edc99887a849e1b7cd9 P 2b323f8a7603407b8244f42c121688a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:09:36.926690 19458 tablet_replica.cc:442] T d746dd90ea924edc99887a849e1b7cd9 P 8b6b057ca85642cc8f73cb7dbf846a49: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:09:36.927345 19495 mvcc.cc:205] Tried to move back new op lower bound from 7315928583831973888 to 7315928583530328064. Current Snapshot: MvccSnapshot[applied={T|T < 7315928583831973888}]
I20260807 15:09:36.929015 19492 mvcc.cc:205] Tried to move back new op lower bound from 7315928583831973888 to 7315928583530328064. Current Snapshot: MvccSnapshot[applied={T|T < 7315928583831973888}]
I20260807 15:09:36.933466 19450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.928863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46408 (local address 127.15.226.65:37213)
0807 15:09:36.929015 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:36.929019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:36.929297 (+ 278us) server_negotiation.cc:407] Connection header received
0807 15:09:36.929500 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:36.929505 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:36.929562 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:36.929676 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:09:36.930011 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.930813 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.931977 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.932217 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.932337 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.932623 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:09:36.933050 (+ 427us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:09:36.933182 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:36.933255 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 15:09:36.933310 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 15:09:36.948418 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.944303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:41341 (local address 127.15.226.67:45979)
0807 15:09:36.944457 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:36.944460 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:36.944617 (+ 157us) server_negotiation.cc:407] Connection header received
0807 15:09:36.944775 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:36.944777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:36.944825 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:36.944899 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:09:36.945296 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.945836 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.947184 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.947474 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.947560 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.947963 (+ 403us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:09:36.948044 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:09:36.948122 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:36.948259 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 15:09:36.948309 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 15:09:36.948508 19505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.944116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:45979 (local address 127.15.226.65:41341)
0807 15:09:36.944504 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:36.944523 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:36.944652 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:36.944940 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:36.944943 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:36.944952 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:09:36.945124 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.945132 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.945967 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.945971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.947045 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.947058 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.947602 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.947606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.947711 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.947772 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:09:36.948144 (+ 372us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:09:36.948151 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 15:09:36.948236 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 15:09:36.948328 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":145,"threads_started":1}
I20260807 15:09:36.956449 19151 raft_consensus.cc:1261] T a049f60ecc1444058f0a2c310334d284 P 2b323f8a7603407b8244f42c121688a2 [term 1 FOLLOWER]: Refusing update from remote peer 8b6b057ca85642cc8f73cb7dbf846a49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:36.956862 19356 raft_consensus.cc:1261] T a049f60ecc1444058f0a2c310334d284 P 06edd948f3f14a619abe46175f7faa00 [term 1 FOLLOWER]: Refusing update from remote peer 8b6b057ca85642cc8f73cb7dbf846a49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:36.957005 19458 consensus_queue.cc:1035] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:36.957430 19468 consensus_queue.cc:1035] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:36.962045 19490 mvcc.cc:205] Tried to move back new op lower bound from 7315928584008220672 to 7315928581989396480. Current Snapshot: MvccSnapshot[applied={T|T < 7315928584008220672}]
I20260807 15:09:36.983683 19450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.979371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:35847 (local address 127.15.226.65:37213)
0807 15:09:36.979544 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:36.979549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:36.979580 (+ 31us) server_negotiation.cc:407] Connection header received
0807 15:09:36.979630 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:36.979633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:36.979692 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:36.979799 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:09:36.980157 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.980965 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.982345 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.982571 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.982679 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.983095 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:09:36.983187 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:09:36.983268 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:36.983460 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 15:09:36.983532 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 15:09:36.984110 19511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:36.979012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:37213 (local address 127.15.226.67:35847)
0807 15:09:36.979396 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:36.979416 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:36.979524 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:36.979816 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:36.979819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:36.979872 (+ 53us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:09:36.980026 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.980034 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.981117 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.981121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.982195 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:36.982219 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:36.982741 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:36.982746 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:36.982850 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:36.982910 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:09:36.983346 (+ 436us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:09:36.983352 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:09:36.983873 (+ 521us) client_negotiation.cc:240] Negotiation successful
0807 15:09:36.983965 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":110,"threads_started":1}
I20260807 15:09:37.013692 18720 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:37.013912 18720 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:37.016759 18720 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591: Sending DeleteTablet for 3 replicas of tablet a049f60ecc1444058f0a2c310334d284
I20260807 15:09:37.017894 18915 tablet_service.cc:1514] Processing DeleteTablet for tablet a049f60ecc1444058f0a2c310334d284 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:37 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:46396
I20260807 15:09:37.018788 19119 tablet_service.cc:1514] Processing DeleteTablet for tablet a049f60ecc1444058f0a2c310334d284 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:37 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:42468
I20260807 15:09:37.018818 19331 tablet_service.cc:1514] Processing DeleteTablet for tablet a049f60ecc1444058f0a2c310334d284 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:37 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:46340
I20260807 15:09:37.018869 19515 tablet_replica.cc:330] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49: stopping tablet replica
I20260807 15:09:37.019222 18720 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 15:09:37.019367 19515 raft_consensus.cc:2229] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 1 LEADER]: Raft consensus shutting down.
W20260807 15:09:37.019739 18720 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:09:37.020098 19515 raft_consensus.cc:2258] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:37.021628 19515 ts_tablet_manager.cc:1905] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:37.023901 19516 tablet_replica.cc:330] T a049f60ecc1444058f0a2c310334d284 P 2b323f8a7603407b8244f42c121688a2: stopping tablet replica
I20260807 15:09:37.024187 19517 tablet_replica.cc:330] T a049f60ecc1444058f0a2c310334d284 P 06edd948f3f14a619abe46175f7faa00: stopping tablet replica
I20260807 15:09:37.024586 19517 raft_consensus.cc:2229] T a049f60ecc1444058f0a2c310334d284 P 06edd948f3f14a619abe46175f7faa00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:37.024937 19516 raft_consensus.cc:2229] T a049f60ecc1444058f0a2c310334d284 P 2b323f8a7603407b8244f42c121688a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:37.025151 19119 tablet_service.cc:1467] Processing CreateTablet for tablet f53b650ef89d4a5f84f00fa0d88cabf3 (DEFAULT_TABLE table=test-table [id=03986e0741bd454493ca191f36716d26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:37.025456 19517 raft_consensus.cc:2258] T a049f60ecc1444058f0a2c310334d284 P 06edd948f3f14a619abe46175f7faa00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:37.025499 19119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f53b650ef89d4a5f84f00fa0d88cabf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:37.025235 19516 raft_consensus.cc:2258] T a049f60ecc1444058f0a2c310334d284 P 2b323f8a7603407b8244f42c121688a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:37.025655 18915 tablet_service.cc:1467] Processing CreateTablet for tablet f53b650ef89d4a5f84f00fa0d88cabf3 (DEFAULT_TABLE table=test-table [id=03986e0741bd454493ca191f36716d26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:37.026242 19331 tablet_service.cc:1467] Processing CreateTablet for tablet f53b650ef89d4a5f84f00fa0d88cabf3 (DEFAULT_TABLE table=test-table [id=03986e0741bd454493ca191f36716d26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:37.026515 19331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f53b650ef89d4a5f84f00fa0d88cabf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:37.027432 19516 ts_tablet_manager.cc:1905] T a049f60ecc1444058f0a2c310334d284 P 2b323f8a7603407b8244f42c121688a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:37.028112 19517 ts_tablet_manager.cc:1905] T a049f60ecc1444058f0a2c310334d284 P 06edd948f3f14a619abe46175f7faa00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:37.028205 19453 tablet_bootstrap.cc:492] T f53b650ef89d4a5f84f00fa0d88cabf3 P 2b323f8a7603407b8244f42c121688a2: Bootstrap starting.
I20260807 15:09:37.029969 19454 tablet_bootstrap.cc:492] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00: Bootstrap starting.
I20260807 15:09:37.030598 19453 tablet_bootstrap.cc:654] T f53b650ef89d4a5f84f00fa0d88cabf3 P 2b323f8a7603407b8244f42c121688a2: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:37.030781 19454 tablet_bootstrap.cc:654] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:37.031939 19453 tablet_bootstrap.cc:492] T f53b650ef89d4a5f84f00fa0d88cabf3 P 2b323f8a7603407b8244f42c121688a2: No bootstrap required, opened a new log
I20260807 15:09:37.032027 19453 ts_tablet_manager.cc:1397] T f53b650ef89d4a5f84f00fa0d88cabf3 P 2b323f8a7603407b8244f42c121688a2: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 15:09:37.032495 19453 raft_consensus.cc:348] T f53b650ef89d4a5f84f00fa0d88cabf3 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:37.032614 19453 raft_consensus.cc:374] T f53b650ef89d4a5f84f00fa0d88cabf3 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:37.032652 19453 raft_consensus.cc:729] T f53b650ef89d4a5f84f00fa0d88cabf3 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b323f8a7603407b8244f42c121688a2, State: Initialized, Role: FOLLOWER
I20260807 15:09:37.032768 19453 consensus_queue.cc:260] T f53b650ef89d4a5f84f00fa0d88cabf3 P 2b323f8a7603407b8244f42c121688a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:37.033039 19453 ts_tablet_manager.cc:1428] T f53b650ef89d4a5f84f00fa0d88cabf3 P 2b323f8a7603407b8244f42c121688a2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:09:37.034055 19454 tablet_bootstrap.cc:492] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00: No bootstrap required, opened a new log
I20260807 15:09:37.034150 19454 ts_tablet_manager.cc:1397] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260807 15:09:37.034591 19454 raft_consensus.cc:348] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:37.034739 19454 raft_consensus.cc:374] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:37.034775 19454 raft_consensus.cc:729] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06edd948f3f14a619abe46175f7faa00, State: Initialized, Role: FOLLOWER
I20260807 15:09:37.034895 19454 consensus_queue.cc:260] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:37.035164 19454 ts_tablet_manager.cc:1428] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:09:37.041256 18915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f53b650ef89d4a5f84f00fa0d88cabf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:37.061606 19516 ts_tablet_manager.cc:1918] T a049f60ecc1444058f0a2c310334d284 P 2b323f8a7603407b8244f42c121688a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:09:37.061789 19516 log.cc:1196] T a049f60ecc1444058f0a2c310334d284 P 2b323f8a7603407b8244f42c121688a2: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/a049f60ecc1444058f0a2c310334d284
I20260807 15:09:37.062180 19516 ts_tablet_manager.cc:1939] T a049f60ecc1444058f0a2c310334d284 P 2b323f8a7603407b8244f42c121688a2: Deleting consensus metadata
I20260807 15:09:37.064006 18702 catalog_manager.cc:4841] TS 2b323f8a7603407b8244f42c121688a2 (127.15.226.66:42023): tablet a049f60ecc1444058f0a2c310334d284 (table test-table [id=f35bd5a878d54b709f399aefff4edd92]) successfully deleted
I20260807 15:09:37.068677 19461 raft_consensus.cc:482] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:09:37.068845 19517 ts_tablet_manager.cc:1918] T a049f60ecc1444058f0a2c310334d284 P 06edd948f3f14a619abe46175f7faa00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:09:37.068892 19461 raft_consensus.cc:504] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:37.068987 19517 log.cc:1196] T a049f60ecc1444058f0a2c310334d284 P 06edd948f3f14a619abe46175f7faa00: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/a049f60ecc1444058f0a2c310334d284
I20260807 15:09:37.069437 19517 ts_tablet_manager.cc:1939] T a049f60ecc1444058f0a2c310334d284 P 06edd948f3f14a619abe46175f7faa00: Deleting consensus metadata
I20260807 15:09:37.069413 19461 leader_election.cc:288] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b6b057ca85642cc8f73cb7dbf846a49 (127.15.226.65:37213), 2b323f8a7603407b8244f42c121688a2 (127.15.226.66:42023)
I20260807 15:09:37.070045 19151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53b650ef89d4a5f84f00fa0d88cabf3" candidate_uuid: "06edd948f3f14a619abe46175f7faa00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b323f8a7603407b8244f42c121688a2" is_pre_election: true
I20260807 15:09:37.070207 19151 raft_consensus.cc:2454] T f53b650ef89d4a5f84f00fa0d88cabf3 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06edd948f3f14a619abe46175f7faa00 in term 0.
I20260807 15:09:37.070550 19268 leader_election.cc:302] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06edd948f3f14a619abe46175f7faa00, 2b323f8a7603407b8244f42c121688a2; no voters:
W20260807 15:09:37.070698 19267 leader_election.cc:341] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8b6b057ca85642cc8f73cb7dbf846a49 (127.15.226.65:37213): Not found: Tablet not found: f53b650ef89d4a5f84f00fa0d88cabf3
I20260807 15:09:37.070878 19461 raft_consensus.cc:2781] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:09:37.070946 19461 raft_consensus.cc:482] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:09:37.070986 19461 raft_consensus.cc:3037] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:37.097600 18700 catalog_manager.cc:4841] TS 06edd948f3f14a619abe46175f7faa00 (127.15.226.67:45979): tablet a049f60ecc1444058f0a2c310334d284 (table test-table [id=f35bd5a878d54b709f399aefff4edd92]) successfully deleted
I20260807 15:09:37.137997 19461 raft_consensus.cc:504] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:37.138132 19515 ts_tablet_manager.cc:1918] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:09:37.138259 19515 log.cc:1196] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-0/wal/wals/a049f60ecc1444058f0a2c310334d284
I20260807 15:09:37.138682 19461 leader_election.cc:288] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00 [CANDIDATE]: Term 1 election: Requested vote from peers 8b6b057ca85642cc8f73cb7dbf846a49 (127.15.226.65:37213), 2b323f8a7603407b8244f42c121688a2 (127.15.226.66:42023)
I20260807 15:09:37.138720 19515 ts_tablet_manager.cc:1939] T a049f60ecc1444058f0a2c310334d284 P 8b6b057ca85642cc8f73cb7dbf846a49: Deleting consensus metadata
W20260807 15:09:37.139662 19267 leader_election.cc:341] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8b6b057ca85642cc8f73cb7dbf846a49 (127.15.226.65:37213): Not found: Tablet not found: f53b650ef89d4a5f84f00fa0d88cabf3
I20260807 15:09:37.140445 18700 catalog_manager.cc:4841] TS 8b6b057ca85642cc8f73cb7dbf846a49 (127.15.226.65:37213): tablet a049f60ecc1444058f0a2c310334d284 (table test-table [id=f35bd5a878d54b709f399aefff4edd92]) successfully deleted
I20260807 15:09:37.141052 19151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f53b650ef89d4a5f84f00fa0d88cabf3" candidate_uuid: "06edd948f3f14a619abe46175f7faa00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b323f8a7603407b8244f42c121688a2"
I20260807 15:09:37.141219 19151 raft_consensus.cc:3037] T f53b650ef89d4a5f84f00fa0d88cabf3 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:37.141723 19521 tablet_bootstrap.cc:492] T f53b650ef89d4a5f84f00fa0d88cabf3 P 8b6b057ca85642cc8f73cb7dbf846a49: Bootstrap starting.
I20260807 15:09:37.142680 19151 raft_consensus.cc:2454] T f53b650ef89d4a5f84f00fa0d88cabf3 P 2b323f8a7603407b8244f42c121688a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06edd948f3f14a619abe46175f7faa00 in term 1.
I20260807 15:09:37.143239 19268 leader_election.cc:302] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 06edd948f3f14a619abe46175f7faa00, 2b323f8a7603407b8244f42c121688a2; no voters: 8b6b057ca85642cc8f73cb7dbf846a49
I20260807 15:09:37.143405 19521 tablet_bootstrap.cc:654] T f53b650ef89d4a5f84f00fa0d88cabf3 P 8b6b057ca85642cc8f73cb7dbf846a49: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:37.143564 19461 raft_consensus.cc:2781] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:37.143663 19461 raft_consensus.cc:686] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00 [term 1 LEADER]: Becoming Leader. State: Replica: 06edd948f3f14a619abe46175f7faa00, State: Running, Role: LEADER
I20260807 15:09:37.143823 19461 consensus_queue.cc:237] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:37.145704 19521 tablet_bootstrap.cc:492] T f53b650ef89d4a5f84f00fa0d88cabf3 P 8b6b057ca85642cc8f73cb7dbf846a49: No bootstrap required, opened a new log
I20260807 15:09:37.145617 18720 catalog_manager.cc:5495] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06edd948f3f14a619abe46175f7faa00 (127.15.226.67). New cstate: current_term: 1 leader_uuid: "06edd948f3f14a619abe46175f7faa00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } health_report { overall_health: HEALTHY } } }
I20260807 15:09:37.145824 19521 ts_tablet_manager.cc:1397] T f53b650ef89d4a5f84f00fa0d88cabf3 P 8b6b057ca85642cc8f73cb7dbf846a49: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260807 15:09:37.146343 19521 raft_consensus.cc:348] T f53b650ef89d4a5f84f00fa0d88cabf3 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:37.146488 19521 raft_consensus.cc:374] T f53b650ef89d4a5f84f00fa0d88cabf3 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:37.146526 19521 raft_consensus.cc:729] T f53b650ef89d4a5f84f00fa0d88cabf3 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b6b057ca85642cc8f73cb7dbf846a49, State: Initialized, Role: FOLLOWER
I20260807 15:09:37.146687 19521 consensus_queue.cc:260] T f53b650ef89d4a5f84f00fa0d88cabf3 P 8b6b057ca85642cc8f73cb7dbf846a49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:37.147037 19521 ts_tablet_manager.cc:1428] T f53b650ef89d4a5f84f00fa0d88cabf3 P 8b6b057ca85642cc8f73cb7dbf846a49: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:09:37.175276 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:37.171318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46352 (local address 127.15.226.67:45979)
0807 15:09:37.171466 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:37.171470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:37.171518 (+ 48us) server_negotiation.cc:407] Connection header received
0807 15:09:37.171784 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:37.171787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:37.171853 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:37.171927 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:09:37.172299 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:37.172832 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:37.174045 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:37.174287 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:37.174381 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:37.174757 (+ 376us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:09:37.174850 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:09:37.174930 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:37.175067 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 15:09:37.175121 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 15:09:37.186316 19511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:37.182398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:45979 (local address 127.15.226.67:54729)
0807 15:09:37.182549 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:37.182565 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:37.182686 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:37.183009 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:37.183011 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:37.183016 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:09:37.183127 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:37.183131 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:37.184291 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:37.184294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:37.184921 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:37.184930 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:37.185539 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:37.185542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:37.185632 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:37.185660 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:09:37.186011 (+ 351us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:09:37.186017 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:09:37.186071 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 15:09:37.186155 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":18}
I20260807 15:09:37.186465 19446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:37.182490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:54729 (local address 127.15.226.67:45979)
0807 15:09:37.182605 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:37.182609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:37.182624 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:09:37.182782 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:37.182786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:37.182852 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:37.183019 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:09:37.183198 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:37.184128 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:37.185060 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:37.185391 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:37.185499 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:37.185809 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:09:37.185904 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:09:37.186001 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:37.186085 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 15:09:37.186189 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":32}
I20260807 15:09:37.193243 18944 raft_consensus.cc:3037] T f53b650ef89d4a5f84f00fa0d88cabf3 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:37.193323 19151 raft_consensus.cc:1261] T f53b650ef89d4a5f84f00fa0d88cabf3 P 2b323f8a7603407b8244f42c121688a2 [term 1 FOLLOWER]: Refusing update from remote peer 06edd948f3f14a619abe46175f7faa00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:37.193800 19484 consensus_queue.cc:1035] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 15:09:37.210542 19490 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 15:09:37.241617 18944 raft_consensus.cc:1261] T f53b650ef89d4a5f84f00fa0d88cabf3 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 1 FOLLOWER]: Refusing update from remote peer 06edd948f3f14a619abe46175f7faa00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:37.242105 19461 consensus_queue.cc:1035] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:38.777448 18720 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:38.777621 18720 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:38.780714 18720 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591: Sending DeleteTablet for 3 replicas of tablet f53b650ef89d4a5f84f00fa0d88cabf3
I20260807 15:09:38.781446 18915 tablet_service.cc:1514] Processing DeleteTablet for tablet f53b650ef89d4a5f84f00fa0d88cabf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:38 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:46396
I20260807 15:09:38.781488 19119 tablet_service.cc:1514] Processing DeleteTablet for tablet f53b650ef89d4a5f84f00fa0d88cabf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:38 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:42468
I20260807 15:09:38.781646 19333 tablet_service.cc:1514] Processing DeleteTablet for tablet f53b650ef89d4a5f84f00fa0d88cabf3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:38 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:46340
I20260807 15:09:38.781895 19555 tablet_replica.cc:330] T f53b650ef89d4a5f84f00fa0d88cabf3 P 2b323f8a7603407b8244f42c121688a2: stopping tablet replica
I20260807 15:09:38.782058 19555 raft_consensus.cc:2229] T f53b650ef89d4a5f84f00fa0d88cabf3 P 2b323f8a7603407b8244f42c121688a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:38.782209 19555 raft_consensus.cc:2258] T f53b650ef89d4a5f84f00fa0d88cabf3 P 2b323f8a7603407b8244f42c121688a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:38.782469 19556 tablet_replica.cc:330] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00: stopping tablet replica
I20260807 15:09:38.782574 19556 raft_consensus.cc:2229] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:09:38.782928 19556 raft_consensus.cc:2258] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:38.783911 19555 ts_tablet_manager.cc:1905] T f53b650ef89d4a5f84f00fa0d88cabf3 P 2b323f8a7603407b8244f42c121688a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:38.786357 19554 tablet_replica.cc:330] T f53b650ef89d4a5f84f00fa0d88cabf3 P 8b6b057ca85642cc8f73cb7dbf846a49: stopping tablet replica
I20260807 15:09:38.786538 19554 raft_consensus.cc:2229] T f53b650ef89d4a5f84f00fa0d88cabf3 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:38.786543 19556 ts_tablet_manager.cc:1905] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:38.786722 19554 raft_consensus.cc:2258] T f53b650ef89d4a5f84f00fa0d88cabf3 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:38.787561 19554 ts_tablet_manager.cc:1905] T f53b650ef89d4a5f84f00fa0d88cabf3 P 8b6b057ca85642cc8f73cb7dbf846a49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:38.787896 19555 ts_tablet_manager.cc:1918] T f53b650ef89d4a5f84f00fa0d88cabf3 P 2b323f8a7603407b8244f42c121688a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 15:09:38.788000 19555 log.cc:1196] T f53b650ef89d4a5f84f00fa0d88cabf3 P 2b323f8a7603407b8244f42c121688a2: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/f53b650ef89d4a5f84f00fa0d88cabf3
I20260807 15:09:38.788347 19555 ts_tablet_manager.cc:1939] T f53b650ef89d4a5f84f00fa0d88cabf3 P 2b323f8a7603407b8244f42c121688a2: Deleting consensus metadata
I20260807 15:09:38.789383 18702 catalog_manager.cc:4841] TS 2b323f8a7603407b8244f42c121688a2 (127.15.226.66:42023): tablet f53b650ef89d4a5f84f00fa0d88cabf3 (table test-table [id=03986e0741bd454493ca191f36716d26]) successfully deleted
I20260807 15:09:38.790185 19556 ts_tablet_manager.cc:1918] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 15:09:38.790289 19556 log.cc:1196] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/f53b650ef89d4a5f84f00fa0d88cabf3
I20260807 15:09:38.790678 19556 ts_tablet_manager.cc:1939] T f53b650ef89d4a5f84f00fa0d88cabf3 P 06edd948f3f14a619abe46175f7faa00: Deleting consensus metadata
I20260807 15:09:38.790846 19554 ts_tablet_manager.cc:1918] T f53b650ef89d4a5f84f00fa0d88cabf3 P 8b6b057ca85642cc8f73cb7dbf846a49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 15:09:38.790944 19554 log.cc:1196] T f53b650ef89d4a5f84f00fa0d88cabf3 P 8b6b057ca85642cc8f73cb7dbf846a49: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-0/wal/wals/f53b650ef89d4a5f84f00fa0d88cabf3
I20260807 15:09:38.791222 19554 ts_tablet_manager.cc:1939] T f53b650ef89d4a5f84f00fa0d88cabf3 P 8b6b057ca85642cc8f73cb7dbf846a49: Deleting consensus metadata
I20260807 15:09:38.792066 18700 catalog_manager.cc:4841] TS 06edd948f3f14a619abe46175f7faa00 (127.15.226.67:45979): tablet f53b650ef89d4a5f84f00fa0d88cabf3 (table test-table [id=03986e0741bd454493ca191f36716d26]) successfully deleted
I20260807 15:09:38.792280 18700 catalog_manager.cc:4841] TS 8b6b057ca85642cc8f73cb7dbf846a49 (127.15.226.65:37213): tablet f53b650ef89d4a5f84f00fa0d88cabf3 (table test-table [id=03986e0741bd454493ca191f36716d26]) successfully deleted
Aug 07 15:09:38 dist-test-slave-3kk6 krb5kdc[18658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115378, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 15:09:38 dist-test-slave-3kk6 krb5kdc[18658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115378, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:38.824587 19570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:38.807966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58824 (local address 127.15.226.126:35401)
0807 15:09:38.808475 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:38.808483 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:38.808499 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:09:38.808569 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:38.808573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:38.808629 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:38.808732 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:38.809559 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.810161 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.811098 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:38.811344 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:38.811455 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:38.820696 (+ 9241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:38.820702 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:38.820705 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:38.820809 (+ 104us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:38.822480 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:38.823076 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:38.823080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:38.823081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:38.823191 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:38.823496 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:38.823498 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:38.823500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:38.823796 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:38.823927 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:38.824172 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 15:09:38.824360 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":327,"threads_started":1}
I20260807 15:09:38.827708 18720 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58824:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:09:38.828290 18720 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:09:38.833787 18915 tablet_service.cc:1467] Processing CreateTablet for tablet 957fb4b2a9974d779d5c9e1a103c4968 (DEFAULT_TABLE table=test-table [id=73fa5f4db33c4ed990d8f932b9645dc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:38.833897 19119 tablet_service.cc:1467] Processing CreateTablet for tablet 957fb4b2a9974d779d5c9e1a103c4968 (DEFAULT_TABLE table=test-table [id=73fa5f4db33c4ed990d8f932b9645dc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:38.833985 19333 tablet_service.cc:1467] Processing CreateTablet for tablet 957fb4b2a9974d779d5c9e1a103c4968 (DEFAULT_TABLE table=test-table [id=73fa5f4db33c4ed990d8f932b9645dc8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:38.834153 18915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 957fb4b2a9974d779d5c9e1a103c4968. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:38.834246 19119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 957fb4b2a9974d779d5c9e1a103c4968. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:38.834300 19333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 957fb4b2a9974d779d5c9e1a103c4968. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:38.837106 19571 tablet_bootstrap.cc:492] T 957fb4b2a9974d779d5c9e1a103c4968 P 2b323f8a7603407b8244f42c121688a2: Bootstrap starting.
I20260807 15:09:38.837453 19573 tablet_bootstrap.cc:492] T 957fb4b2a9974d779d5c9e1a103c4968 P 8b6b057ca85642cc8f73cb7dbf846a49: Bootstrap starting.
I20260807 15:09:38.838164 19571 tablet_bootstrap.cc:654] T 957fb4b2a9974d779d5c9e1a103c4968 P 2b323f8a7603407b8244f42c121688a2: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:38.838836 19573 tablet_bootstrap.cc:654] T 957fb4b2a9974d779d5c9e1a103c4968 P 8b6b057ca85642cc8f73cb7dbf846a49: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:38.839197 19572 tablet_bootstrap.cc:492] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00: Bootstrap starting.
I20260807 15:09:38.840165 19572 tablet_bootstrap.cc:654] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:38.840407 19573 tablet_bootstrap.cc:492] T 957fb4b2a9974d779d5c9e1a103c4968 P 8b6b057ca85642cc8f73cb7dbf846a49: No bootstrap required, opened a new log
I20260807 15:09:38.840515 19573 ts_tablet_manager.cc:1397] T 957fb4b2a9974d779d5c9e1a103c4968 P 8b6b057ca85642cc8f73cb7dbf846a49: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 15:09:38.841033 19571 tablet_bootstrap.cc:492] T 957fb4b2a9974d779d5c9e1a103c4968 P 2b323f8a7603407b8244f42c121688a2: No bootstrap required, opened a new log
I20260807 15:09:38.841162 19571 ts_tablet_manager.cc:1397] T 957fb4b2a9974d779d5c9e1a103c4968 P 2b323f8a7603407b8244f42c121688a2: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260807 15:09:38.841097 19573 raft_consensus.cc:348] T 957fb4b2a9974d779d5c9e1a103c4968 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:38.841256 19573 raft_consensus.cc:374] T 957fb4b2a9974d779d5c9e1a103c4968 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:38.841307 19573 raft_consensus.cc:729] T 957fb4b2a9974d779d5c9e1a103c4968 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b6b057ca85642cc8f73cb7dbf846a49, State: Initialized, Role: FOLLOWER
I20260807 15:09:38.841495 19572 tablet_bootstrap.cc:492] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00: No bootstrap required, opened a new log
I20260807 15:09:38.841471 19573 consensus_queue.cc:260] T 957fb4b2a9974d779d5c9e1a103c4968 P 8b6b057ca85642cc8f73cb7dbf846a49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:38.841593 19572 ts_tablet_manager.cc:1397] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:09:38.841678 19571 raft_consensus.cc:348] T 957fb4b2a9974d779d5c9e1a103c4968 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:38.841801 19571 raft_consensus.cc:374] T 957fb4b2a9974d779d5c9e1a103c4968 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:38.841842 19571 raft_consensus.cc:729] T 957fb4b2a9974d779d5c9e1a103c4968 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b323f8a7603407b8244f42c121688a2, State: Initialized, Role: FOLLOWER
I20260807 15:09:38.841965 19573 ts_tablet_manager.cc:1428] T 957fb4b2a9974d779d5c9e1a103c4968 P 8b6b057ca85642cc8f73cb7dbf846a49: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260807 15:09:38.842007 19571 consensus_queue.cc:260] T 957fb4b2a9974d779d5c9e1a103c4968 P 2b323f8a7603407b8244f42c121688a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:38.842056 19572 raft_consensus.cc:348] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:38.842162 19572 raft_consensus.cc:374] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:38.842209 19572 raft_consensus.cc:729] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06edd948f3f14a619abe46175f7faa00, State: Initialized, Role: FOLLOWER
I20260807 15:09:38.842366 19572 consensus_queue.cc:260] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:38.842465 19571 ts_tablet_manager.cc:1428] T 957fb4b2a9974d779d5c9e1a103c4968 P 2b323f8a7603407b8244f42c121688a2: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260807 15:09:38.842798 19572 ts_tablet_manager.cc:1428] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 15:09:38.965956 19539 raft_consensus.cc:482] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:09:38.966176 19539 raft_consensus.cc:504] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:38.966725 19539 leader_election.cc:288] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b6b057ca85642cc8f73cb7dbf846a49 (127.15.226.65:37213), 2b323f8a7603407b8244f42c121688a2 (127.15.226.66:42023)
I20260807 15:09:38.967303 18944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "957fb4b2a9974d779d5c9e1a103c4968" candidate_uuid: "06edd948f3f14a619abe46175f7faa00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" is_pre_election: true
I20260807 15:09:38.967303 19151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "957fb4b2a9974d779d5c9e1a103c4968" candidate_uuid: "06edd948f3f14a619abe46175f7faa00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b323f8a7603407b8244f42c121688a2" is_pre_election: true
I20260807 15:09:38.967500 18944 raft_consensus.cc:2454] T 957fb4b2a9974d779d5c9e1a103c4968 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06edd948f3f14a619abe46175f7faa00 in term 0.
I20260807 15:09:38.967518 19151 raft_consensus.cc:2454] T 957fb4b2a9974d779d5c9e1a103c4968 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06edd948f3f14a619abe46175f7faa00 in term 0.
I20260807 15:09:38.967927 19268 leader_election.cc:302] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06edd948f3f14a619abe46175f7faa00, 2b323f8a7603407b8244f42c121688a2; no voters:
I20260807 15:09:38.968145 19539 raft_consensus.cc:2781] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:09:38.968215 19539 raft_consensus.cc:482] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:09:38.968245 19539 raft_consensus.cc:3037] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:38.969350 19539 raft_consensus.cc:504] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:38.969749 19539 leader_election.cc:288] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00 [CANDIDATE]: Term 1 election: Requested vote from peers 8b6b057ca85642cc8f73cb7dbf846a49 (127.15.226.65:37213), 2b323f8a7603407b8244f42c121688a2 (127.15.226.66:42023)
I20260807 15:09:38.970216 19151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "957fb4b2a9974d779d5c9e1a103c4968" candidate_uuid: "06edd948f3f14a619abe46175f7faa00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b323f8a7603407b8244f42c121688a2"
I20260807 15:09:38.970219 18944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "957fb4b2a9974d779d5c9e1a103c4968" candidate_uuid: "06edd948f3f14a619abe46175f7faa00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b6b057ca85642cc8f73cb7dbf846a49"
I20260807 15:09:38.970374 18944 raft_consensus.cc:3037] T 957fb4b2a9974d779d5c9e1a103c4968 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:38.970405 19151 raft_consensus.cc:3037] T 957fb4b2a9974d779d5c9e1a103c4968 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:38.971601 18944 raft_consensus.cc:2454] T 957fb4b2a9974d779d5c9e1a103c4968 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06edd948f3f14a619abe46175f7faa00 in term 1.
I20260807 15:09:38.971654 19151 raft_consensus.cc:2454] T 957fb4b2a9974d779d5c9e1a103c4968 P 2b323f8a7603407b8244f42c121688a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06edd948f3f14a619abe46175f7faa00 in term 1.
I20260807 15:09:38.972014 19267 leader_election.cc:302] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06edd948f3f14a619abe46175f7faa00, 8b6b057ca85642cc8f73cb7dbf846a49; no voters:
I20260807 15:09:38.972262 19539 raft_consensus.cc:2781] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:38.972390 19539 raft_consensus.cc:686] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00 [term 1 LEADER]: Becoming Leader. State: Replica: 06edd948f3f14a619abe46175f7faa00, State: Running, Role: LEADER
I20260807 15:09:38.972568 19539 consensus_queue.cc:237] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:38.974730 18720 catalog_manager.cc:5495] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06edd948f3f14a619abe46175f7faa00 (127.15.226.67). New cstate: current_term: 1 leader_uuid: "06edd948f3f14a619abe46175f7faa00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } health_report { overall_health: HEALTHY } } }
I20260807 15:09:39.004045 19585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:39.000520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47546 (local address 127.15.226.67:45979)
0807 15:09:39.000871 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:39.000884 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:39.000898 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:09:39.000952 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:39.000954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:39.000999 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:39.001094 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:09:39.001368 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:39.001970 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:39.002889 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:39.003154 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:39.003266 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:39.003551 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:09:39.003641 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:09:39.003725 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:39.003820 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 15:09:39.003875 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":160,"threads_started":1}
I20260807 15:09:39.010746 18944 raft_consensus.cc:1261] T 957fb4b2a9974d779d5c9e1a103c4968 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 1 FOLLOWER]: Refusing update from remote peer 06edd948f3f14a619abe46175f7faa00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:39.010793 19151 raft_consensus.cc:1261] T 957fb4b2a9974d779d5c9e1a103c4968 P 2b323f8a7603407b8244f42c121688a2 [term 1 FOLLOWER]: Refusing update from remote peer 06edd948f3f14a619abe46175f7faa00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:39.011261 19539 consensus_queue.cc:1035] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:39.011420 19534 consensus_queue.cc:1035] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:39.049547 18718 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:39.049721 18718 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:39.052212 18718 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591: Sending DeleteTablet for 3 replicas of tablet 957fb4b2a9974d779d5c9e1a103c4968
I20260807 15:09:39.053013 18915 tablet_service.cc:1514] Processing DeleteTablet for tablet 957fb4b2a9974d779d5c9e1a103c4968 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:39 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:46396
I20260807 15:09:39.053056 19333 tablet_service.cc:1514] Processing DeleteTablet for tablet 957fb4b2a9974d779d5c9e1a103c4968 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:39 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:46340
I20260807 15:09:39.053306 19119 tablet_service.cc:1514] Processing DeleteTablet for tablet 957fb4b2a9974d779d5c9e1a103c4968 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:39 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:42468
I20260807 15:09:39.053622 19554 tablet_replica.cc:330] T 957fb4b2a9974d779d5c9e1a103c4968 P 8b6b057ca85642cc8f73cb7dbf846a49: stopping tablet replica
I20260807 15:09:39.053630 19555 tablet_replica.cc:330] T 957fb4b2a9974d779d5c9e1a103c4968 P 2b323f8a7603407b8244f42c121688a2: stopping tablet replica
I20260807 15:09:39.053781 19554 raft_consensus.cc:2229] T 957fb4b2a9974d779d5c9e1a103c4968 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:39.053824 19555 raft_consensus.cc:2229] T 957fb4b2a9974d779d5c9e1a103c4968 P 2b323f8a7603407b8244f42c121688a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:39.053937 19554 raft_consensus.cc:2258] T 957fb4b2a9974d779d5c9e1a103c4968 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:39.054018 19555 raft_consensus.cc:2258] T 957fb4b2a9974d779d5c9e1a103c4968 P 2b323f8a7603407b8244f42c121688a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:39.053880 18718 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58824:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 15:09:39.054149 19556 tablet_replica.cc:330] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00: stopping tablet replica
I20260807 15:09:39.054266 19556 raft_consensus.cc:2229] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00 [term 1 LEADER]: Raft consensus shutting down.
W20260807 15:09:39.054405 18718 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:09:39.054502 19556 raft_consensus.cc:2258] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:39.055554 19556 ts_tablet_manager.cc:1905] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:39.055570 19555 ts_tablet_manager.cc:1905] T 957fb4b2a9974d779d5c9e1a103c4968 P 2b323f8a7603407b8244f42c121688a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:39.058247 19555 ts_tablet_manager.cc:1918] T 957fb4b2a9974d779d5c9e1a103c4968 P 2b323f8a7603407b8244f42c121688a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:09:39.058341 19555 log.cc:1196] T 957fb4b2a9974d779d5c9e1a103c4968 P 2b323f8a7603407b8244f42c121688a2: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/957fb4b2a9974d779d5c9e1a103c4968
I20260807 15:09:39.058398 19556 ts_tablet_manager.cc:1918] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:09:39.058481 19556 log.cc:1196] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/957fb4b2a9974d779d5c9e1a103c4968
I20260807 15:09:39.058637 19555 ts_tablet_manager.cc:1939] T 957fb4b2a9974d779d5c9e1a103c4968 P 2b323f8a7603407b8244f42c121688a2: Deleting consensus metadata
I20260807 15:09:39.058849 19556 ts_tablet_manager.cc:1939] T 957fb4b2a9974d779d5c9e1a103c4968 P 06edd948f3f14a619abe46175f7faa00: Deleting consensus metadata
I20260807 15:09:39.059486 18702 catalog_manager.cc:4841] TS 2b323f8a7603407b8244f42c121688a2 (127.15.226.66:42023): tablet 957fb4b2a9974d779d5c9e1a103c4968 (table test-table [id=73fa5f4db33c4ed990d8f932b9645dc8]) successfully deleted
I20260807 15:09:39.060019 18700 catalog_manager.cc:4841] TS 06edd948f3f14a619abe46175f7faa00 (127.15.226.67:45979): tablet 957fb4b2a9974d779d5c9e1a103c4968 (table test-table [id=73fa5f4db33c4ed990d8f932b9645dc8]) successfully deleted
I20260807 15:09:39.060225 19554 ts_tablet_manager.cc:1905] T 957fb4b2a9974d779d5c9e1a103c4968 P 8b6b057ca85642cc8f73cb7dbf846a49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:39.060271 19119 tablet_service.cc:1467] Processing CreateTablet for tablet 566304537c53476a8f4973ff5b0d1be5 (DEFAULT_TABLE table=test-table [id=b81a967e4f874319b2852053bd0d4360]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:39.060359 18915 tablet_service.cc:1467] Processing CreateTablet for tablet 566304537c53476a8f4973ff5b0d1be5 (DEFAULT_TABLE table=test-table [id=b81a967e4f874319b2852053bd0d4360]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:39.060546 19119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 566304537c53476a8f4973ff5b0d1be5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:39.060608 18915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 566304537c53476a8f4973ff5b0d1be5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:39.060935 19333 tablet_service.cc:1467] Processing CreateTablet for tablet 566304537c53476a8f4973ff5b0d1be5 (DEFAULT_TABLE table=test-table [id=b81a967e4f874319b2852053bd0d4360]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:39.061231 19333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 566304537c53476a8f4973ff5b0d1be5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:39.062924 19571 tablet_bootstrap.cc:492] T 566304537c53476a8f4973ff5b0d1be5 P 2b323f8a7603407b8244f42c121688a2: Bootstrap starting.
I20260807 15:09:39.062973 19573 tablet_bootstrap.cc:492] T 566304537c53476a8f4973ff5b0d1be5 P 8b6b057ca85642cc8f73cb7dbf846a49: Bootstrap starting.
I20260807 15:09:39.063714 19573 tablet_bootstrap.cc:654] T 566304537c53476a8f4973ff5b0d1be5 P 8b6b057ca85642cc8f73cb7dbf846a49: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:39.063932 19572 tablet_bootstrap.cc:492] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00: Bootstrap starting.
I20260807 15:09:39.063990 19571 tablet_bootstrap.cc:654] T 566304537c53476a8f4973ff5b0d1be5 P 2b323f8a7603407b8244f42c121688a2: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:39.064778 19572 tablet_bootstrap.cc:654] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:39.064953 19573 tablet_bootstrap.cc:492] T 566304537c53476a8f4973ff5b0d1be5 P 8b6b057ca85642cc8f73cb7dbf846a49: No bootstrap required, opened a new log
I20260807 15:09:39.065069 19573 ts_tablet_manager.cc:1397] T 566304537c53476a8f4973ff5b0d1be5 P 8b6b057ca85642cc8f73cb7dbf846a49: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 15:09:39.065106 19554 ts_tablet_manager.cc:1918] T 957fb4b2a9974d779d5c9e1a103c4968 P 8b6b057ca85642cc8f73cb7dbf846a49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:09:39.065196 19554 log.cc:1196] T 957fb4b2a9974d779d5c9e1a103c4968 P 8b6b057ca85642cc8f73cb7dbf846a49: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-0/wal/wals/957fb4b2a9974d779d5c9e1a103c4968
I20260807 15:09:39.065493 19554 ts_tablet_manager.cc:1939] T 957fb4b2a9974d779d5c9e1a103c4968 P 8b6b057ca85642cc8f73cb7dbf846a49: Deleting consensus metadata
I20260807 15:09:39.065596 19573 raft_consensus.cc:348] T 566304537c53476a8f4973ff5b0d1be5 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:39.065739 19573 raft_consensus.cc:374] T 566304537c53476a8f4973ff5b0d1be5 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:39.065781 19573 raft_consensus.cc:729] T 566304537c53476a8f4973ff5b0d1be5 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b6b057ca85642cc8f73cb7dbf846a49, State: Initialized, Role: FOLLOWER
I20260807 15:09:39.066030 19572 tablet_bootstrap.cc:492] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00: No bootstrap required, opened a new log
I20260807 15:09:39.066128 19572 ts_tablet_manager.cc:1397] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:09:39.066030 19573 consensus_queue.cc:260] T 566304537c53476a8f4973ff5b0d1be5 P 8b6b057ca85642cc8f73cb7dbf846a49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:39.066432 19573 ts_tablet_manager.cc:1428] T 566304537c53476a8f4973ff5b0d1be5 P 8b6b057ca85642cc8f73cb7dbf846a49: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260807 15:09:39.066565 19572 raft_consensus.cc:348] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:39.066720 19572 raft_consensus.cc:374] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:39.066779 19572 raft_consensus.cc:729] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06edd948f3f14a619abe46175f7faa00, State: Initialized, Role: FOLLOWER
I20260807 15:09:39.066900 19572 consensus_queue.cc:260] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:39.067240 19572 ts_tablet_manager.cc:1428] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:09:39.067574 19571 tablet_bootstrap.cc:492] T 566304537c53476a8f4973ff5b0d1be5 P 2b323f8a7603407b8244f42c121688a2: No bootstrap required, opened a new log
I20260807 15:09:39.067685 19571 ts_tablet_manager.cc:1397] T 566304537c53476a8f4973ff5b0d1be5 P 2b323f8a7603407b8244f42c121688a2: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260807 15:09:39.067685 18700 catalog_manager.cc:4841] TS 8b6b057ca85642cc8f73cb7dbf846a49 (127.15.226.65:37213): tablet 957fb4b2a9974d779d5c9e1a103c4968 (table test-table [id=73fa5f4db33c4ed990d8f932b9645dc8]) successfully deleted
I20260807 15:09:39.068194 19571 raft_consensus.cc:348] T 566304537c53476a8f4973ff5b0d1be5 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:39.068290 19571 raft_consensus.cc:374] T 566304537c53476a8f4973ff5b0d1be5 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:39.068312 19571 raft_consensus.cc:729] T 566304537c53476a8f4973ff5b0d1be5 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b323f8a7603407b8244f42c121688a2, State: Initialized, Role: FOLLOWER
I20260807 15:09:39.068454 19571 consensus_queue.cc:260] T 566304537c53476a8f4973ff5b0d1be5 P 2b323f8a7603407b8244f42c121688a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:39.068776 19571 ts_tablet_manager.cc:1428] T 566304537c53476a8f4973ff5b0d1be5 P 2b323f8a7603407b8244f42c121688a2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:09:39.371289 19534 raft_consensus.cc:482] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:09:39.371469 19534 raft_consensus.cc:504] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:39.372046 19534 leader_election.cc:288] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b6b057ca85642cc8f73cb7dbf846a49 (127.15.226.65:37213), 2b323f8a7603407b8244f42c121688a2 (127.15.226.66:42023)
I20260807 15:09:39.372740 18944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "566304537c53476a8f4973ff5b0d1be5" candidate_uuid: "06edd948f3f14a619abe46175f7faa00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" is_pre_election: true
I20260807 15:09:39.372751 19150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "566304537c53476a8f4973ff5b0d1be5" candidate_uuid: "06edd948f3f14a619abe46175f7faa00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b323f8a7603407b8244f42c121688a2" is_pre_election: true
I20260807 15:09:39.372932 18944 raft_consensus.cc:2454] T 566304537c53476a8f4973ff5b0d1be5 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06edd948f3f14a619abe46175f7faa00 in term 0.
I20260807 15:09:39.372963 19150 raft_consensus.cc:2454] T 566304537c53476a8f4973ff5b0d1be5 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06edd948f3f14a619abe46175f7faa00 in term 0.
I20260807 15:09:39.373325 19267 leader_election.cc:302] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06edd948f3f14a619abe46175f7faa00, 8b6b057ca85642cc8f73cb7dbf846a49; no voters:
I20260807 15:09:39.373517 19534 raft_consensus.cc:2781] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:09:39.373586 19534 raft_consensus.cc:482] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:09:39.373629 19534 raft_consensus.cc:3037] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:39.374859 19534 raft_consensus.cc:504] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:39.375298 19534 leader_election.cc:288] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00 [CANDIDATE]: Term 1 election: Requested vote from peers 8b6b057ca85642cc8f73cb7dbf846a49 (127.15.226.65:37213), 2b323f8a7603407b8244f42c121688a2 (127.15.226.66:42023)
I20260807 15:09:39.375718 18944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "566304537c53476a8f4973ff5b0d1be5" candidate_uuid: "06edd948f3f14a619abe46175f7faa00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b6b057ca85642cc8f73cb7dbf846a49"
I20260807 15:09:39.375783 19150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "566304537c53476a8f4973ff5b0d1be5" candidate_uuid: "06edd948f3f14a619abe46175f7faa00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b323f8a7603407b8244f42c121688a2"
I20260807 15:09:39.375844 18944 raft_consensus.cc:3037] T 566304537c53476a8f4973ff5b0d1be5 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:39.375952 19150 raft_consensus.cc:3037] T 566304537c53476a8f4973ff5b0d1be5 P 2b323f8a7603407b8244f42c121688a2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:39.376964 18944 raft_consensus.cc:2454] T 566304537c53476a8f4973ff5b0d1be5 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06edd948f3f14a619abe46175f7faa00 in term 1.
I20260807 15:09:39.376974 19150 raft_consensus.cc:2454] T 566304537c53476a8f4973ff5b0d1be5 P 2b323f8a7603407b8244f42c121688a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06edd948f3f14a619abe46175f7faa00 in term 1.
I20260807 15:09:39.377301 19267 leader_election.cc:302] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06edd948f3f14a619abe46175f7faa00, 8b6b057ca85642cc8f73cb7dbf846a49; no voters:
I20260807 15:09:39.377556 19534 raft_consensus.cc:2781] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:39.377684 19534 raft_consensus.cc:686] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00 [term 1 LEADER]: Becoming Leader. State: Replica: 06edd948f3f14a619abe46175f7faa00, State: Running, Role: LEADER
I20260807 15:09:39.377848 19534 consensus_queue.cc:237] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } }
I20260807 15:09:39.379886 18719 catalog_manager.cc:5495] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06edd948f3f14a619abe46175f7faa00 (127.15.226.67). New cstate: current_term: 1 leader_uuid: "06edd948f3f14a619abe46175f7faa00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06edd948f3f14a619abe46175f7faa00" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 45979 } health_report { overall_health: HEALTHY } } }
I20260807 15:09:39.451301 19150 raft_consensus.cc:1261] T 566304537c53476a8f4973ff5b0d1be5 P 2b323f8a7603407b8244f42c121688a2 [term 1 FOLLOWER]: Refusing update from remote peer 06edd948f3f14a619abe46175f7faa00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:39.451390 18944 raft_consensus.cc:1261] T 566304537c53476a8f4973ff5b0d1be5 P 8b6b057ca85642cc8f73cb7dbf846a49 [term 1 FOLLOWER]: Refusing update from remote peer 06edd948f3f14a619abe46175f7faa00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:39.451779 19534 consensus_queue.cc:1035] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b323f8a7603407b8244f42c121688a2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:39.451984 19598 consensus_queue.cc:1035] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b6b057ca85642cc8f73cb7dbf846a49" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 15:09:39.467027 19531 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 15:09:41.022010 18718 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:41.022177 18718 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:41.024972 18718 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc5e3fa914b44a32ba90256378122591: Sending DeleteTablet for 3 replicas of tablet 566304537c53476a8f4973ff5b0d1be5
I20260807 15:09:41.025681 18915 tablet_service.cc:1514] Processing DeleteTablet for tablet 566304537c53476a8f4973ff5b0d1be5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:41 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:46396
I20260807 15:09:41.025688 19333 tablet_service.cc:1514] Processing DeleteTablet for tablet 566304537c53476a8f4973ff5b0d1be5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:41 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:46340
I20260807 15:09:41.026075 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 18824
I20260807 15:09:41.026048 19119 tablet_service.cc:1514] Processing DeleteTablet for tablet 566304537c53476a8f4973ff5b0d1be5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:41 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:42468
I20260807 15:09:41.026087 19631 tablet_replica.cc:330] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00: stopping tablet replica
I20260807 15:09:41.030089 19633 tablet_replica.cc:330] T 566304537c53476a8f4973ff5b0d1be5 P 2b323f8a7603407b8244f42c121688a2: stopping tablet replica
I20260807 15:09:41.030308 19633 raft_consensus.cc:2229] T 566304537c53476a8f4973ff5b0d1be5 P 2b323f8a7603407b8244f42c121688a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:41.030468 19633 raft_consensus.cc:2258] T 566304537c53476a8f4973ff5b0d1be5 P 2b323f8a7603407b8244f42c121688a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:41.031932 19633 ts_tablet_manager.cc:1905] T 566304537c53476a8f4973ff5b0d1be5 P 2b323f8a7603407b8244f42c121688a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:41.034308 19631 raft_consensus.cc:2229] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:09:41.034612 19631 raft_consensus.cc:2258] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:41.035655 19631 ts_tablet_manager.cc:1905] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:41.036899 19633 ts_tablet_manager.cc:1918] T 566304537c53476a8f4973ff5b0d1be5 P 2b323f8a7603407b8244f42c121688a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 15:09:41.037031 19633 log.cc:1196] T 566304537c53476a8f4973ff5b0d1be5 P 2b323f8a7603407b8244f42c121688a2: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/566304537c53476a8f4973ff5b0d1be5
I20260807 15:09:41.037449 19633 ts_tablet_manager.cc:1939] T 566304537c53476a8f4973ff5b0d1be5 P 2b323f8a7603407b8244f42c121688a2: Deleting consensus metadata
I20260807 15:09:41.038247 19631 ts_tablet_manager.cc:1918] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 15:09:41.038358 19631 log.cc:1196] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/566304537c53476a8f4973ff5b0d1be5
I20260807 15:09:41.038610 18702 catalog_manager.cc:4841] TS 2b323f8a7603407b8244f42c121688a2 (127.15.226.66:42023): tablet 566304537c53476a8f4973ff5b0d1be5 (table test-table [id=b81a967e4f874319b2852053bd0d4360]) successfully deleted
I20260807 15:09:41.038753 19631 ts_tablet_manager.cc:1939] T 566304537c53476a8f4973ff5b0d1be5 P 06edd948f3f14a619abe46175f7faa00: Deleting consensus metadata
I20260807 15:09:41.039533 18700 catalog_manager.cc:4841] TS 06edd948f3f14a619abe46175f7faa00 (127.15.226.67:45979): tablet 566304537c53476a8f4973ff5b0d1be5 (table test-table [id=b81a967e4f874319b2852053bd0d4360]) successfully deleted
W20260807 15:09:41.052032 18700 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.226.65:37213 (error 108)
I20260807 15:09:41.052167 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 19024
I20260807 15:09:41.059235 19635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:41.058477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:37213 (local address 127.0.0.1:40398)
0807 15:09:41.058933 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:41.059065 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.65:37213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":117,"threads_started":1}
W20260807 15:09:41.059656 18700 catalog_manager.cc:4584] TS 8b6b057ca85642cc8f73cb7dbf846a49 (127.15.226.65:37213): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 566304537c53476a8f4973ff5b0d1be5: Network error: Client connection negotiation failed: client connection to 127.15.226.65:37213: connect: Connection refused (error 111)
I20260807 15:09:41.066890 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 19238
I20260807 15:09:41.082491 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 18673
2026-08-07T15:09:41Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (7142 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:09:41 dist-test-slave-3kk6 krb5kdc[19638](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:09:41 dist-test-slave-3kk6 krb5kdc[19638](info): set up 2 sockets
Aug 07 15:09:41 dist-test-slave-3kk6 krb5kdc[19638](info): commencing operation
krb5kdc: starting...
W20260807 15:09:42.342883 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:09:42 dist-test-slave-3kk6 krb5kdc[19638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:09:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:09:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:09:42.490872 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:34183
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:40123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:34183
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:42.611351 19655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:42.611732 19655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:42.611843 19655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:42.616089 19655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:42.616192 19655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:42.616230 19655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:42.616294 19655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:42.616334 19655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:09:42.621887 19655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:40123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:34183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:34183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:42.623202 19655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:42.624346 19655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:42.634044 19663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:42.634073 19665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:42.634109 19655 server_base.cc:1034] running on GCE node
W20260807 15:09:42.634044 19667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:42.635069 19655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:42.636404 19655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:42.637667 19655 hybrid_clock.cc:648] HybridClock initialized: now 1786115382637636 us; error 53 us; skew 500 ppm
Aug 07 15:09:42 dist-test-slave-3kk6 krb5kdc[19638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115382, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:42.640900 19655 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:09:42.641963 19655 webserver.cc:460] Webserver started at http://127.15.226.126:36805/ using document root <none> and password file <none>
I20260807 15:09:42.642606 19655 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:42.642740 19655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:42.642995 19655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:42.645239 19655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "489c497ec92f4d7bbc6541ff9a96ed36"
format_stamp: "Formatted at 2026-08-07 15:09:42 on dist-test-slave-3kk6"
server_key: "9792b563b9bf86ded3068d681e62bfbe"
server_key_iv: "df15365b45b0db86d9fc61d15a011c61"
server_key_version: "encryptionkey@0"
I20260807 15:09:42.645885 19655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "489c497ec92f4d7bbc6541ff9a96ed36"
format_stamp: "Formatted at 2026-08-07 15:09:42 on dist-test-slave-3kk6"
server_key: "9792b563b9bf86ded3068d681e62bfbe"
server_key_iv: "df15365b45b0db86d9fc61d15a011c61"
server_key_version: "encryptionkey@0"
I20260807 15:09:42.650125 19655 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260807 15:09:42.652910 19675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:42.654268 19655 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:09:42.654451 19655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "489c497ec92f4d7bbc6541ff9a96ed36"
format_stamp: "Formatted at 2026-08-07 15:09:42 on dist-test-slave-3kk6"
server_key: "9792b563b9bf86ded3068d681e62bfbe"
server_key_iv: "df15365b45b0db86d9fc61d15a011c61"
server_key_version: "encryptionkey@0"
I20260807 15:09:42.654590 19655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:42.682262 19655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:42.685734 19655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:42.685988 19655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:42.693892 19655 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:34183
I20260807 15:09:42.693921 19743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:34183 every 8 connection(s)
I20260807 15:09:42.695063 19655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:09:42.698035 19744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:42.699558 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 19655
I20260807 15:09:42.699669 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:09:42.700100 16265 external_mini_cluster.cc:1428] Setting key bdb89f499395acf4f92ca74234489594
I20260807 15:09:42.704797 19744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 489c497ec92f4d7bbc6541ff9a96ed36: Bootstrap starting.
I20260807 15:09:42.707307 19744 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 489c497ec92f4d7bbc6541ff9a96ed36: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:42.708110 19744 log.cc:824] T 00000000000000000000000000000000 P 489c497ec92f4d7bbc6541ff9a96ed36: Log is configured to *not* fsync() on all Append() calls
Aug 07 15:09:42 dist-test-slave-3kk6 krb5kdc[19638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115382, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:42.710728 19744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 489c497ec92f4d7bbc6541ff9a96ed36: No bootstrap required, opened a new log
I20260807 15:09:42.714887 19744 raft_consensus.cc:348] T 00000000000000000000000000000000 P 489c497ec92f4d7bbc6541ff9a96ed36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "489c497ec92f4d7bbc6541ff9a96ed36" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34183 } }
I20260807 15:09:42.715176 19744 raft_consensus.cc:374] T 00000000000000000000000000000000 P 489c497ec92f4d7bbc6541ff9a96ed36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:42.715261 19744 raft_consensus.cc:729] T 00000000000000000000000000000000 P 489c497ec92f4d7bbc6541ff9a96ed36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 489c497ec92f4d7bbc6541ff9a96ed36, State: Initialized, Role: FOLLOWER
I20260807 15:09:42.715852 19744 consensus_queue.cc:260] T 00000000000000000000000000000000 P 489c497ec92f4d7bbc6541ff9a96ed36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "489c497ec92f4d7bbc6541ff9a96ed36" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34183 } }
I20260807 15:09:42.716035 19744 raft_consensus.cc:388] T 00000000000000000000000000000000 P 489c497ec92f4d7bbc6541ff9a96ed36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:09:42.716102 19744 raft_consensus.cc:482] T 00000000000000000000000000000000 P 489c497ec92f4d7bbc6541ff9a96ed36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:09:42.716202 19744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 489c497ec92f4d7bbc6541ff9a96ed36 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:42.717020 19749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:42.701701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36308 (local address 127.15.226.126:34183)
0807 15:09:42.702234 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:42.702252 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:42.702293 (+ 41us) server_negotiation.cc:407] Connection header received
0807 15:09:42.703113 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:42.703134 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:42.703503 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:42.703896 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:42.704871 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:42.706041 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:42.707144 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:42.707773 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:42.707996 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:42.711322 (+ 3326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:42.711328 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:42.711344 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:42.711375 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:42.713356 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:42.713883 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:42.713897 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:42.713903 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:42.714034 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:42.714376 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:42.714380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:42.714383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:42.715046 (+ 663us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:42.715318 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:42.715621 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 15:09:42.715959 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":161,"threads_started":1}
I20260807 15:09:42.717540 19744 raft_consensus.cc:504] T 00000000000000000000000000000000 P 489c497ec92f4d7bbc6541ff9a96ed36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "489c497ec92f4d7bbc6541ff9a96ed36" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34183 } }
I20260807 15:09:42.717916 19744 leader_election.cc:302] T 00000000000000000000000000000000 P 489c497ec92f4d7bbc6541ff9a96ed36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 489c497ec92f4d7bbc6541ff9a96ed36; no voters:
I20260807 15:09:42.718286 19744 leader_election.cc:288] T 00000000000000000000000000000000 P 489c497ec92f4d7bbc6541ff9a96ed36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:09:42.718401 19752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 489c497ec92f4d7bbc6541ff9a96ed36 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:42.718644 19752 raft_consensus.cc:686] T 00000000000000000000000000000000 P 489c497ec92f4d7bbc6541ff9a96ed36 [term 1 LEADER]: Becoming Leader. State: Replica: 489c497ec92f4d7bbc6541ff9a96ed36, State: Running, Role: LEADER
I20260807 15:09:42.719066 19752 consensus_queue.cc:237] T 00000000000000000000000000000000 P 489c497ec92f4d7bbc6541ff9a96ed36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "489c497ec92f4d7bbc6541ff9a96ed36" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34183 } }
I20260807 15:09:42.719754 19744 sys_catalog.cc:564] T 00000000000000000000000000000000 P 489c497ec92f4d7bbc6541ff9a96ed36 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:42.720377 19752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 489c497ec92f4d7bbc6541ff9a96ed36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 489c497ec92f4d7bbc6541ff9a96ed36. Latest consensus state: current_term: 1 leader_uuid: "489c497ec92f4d7bbc6541ff9a96ed36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "489c497ec92f4d7bbc6541ff9a96ed36" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34183 } } }
I20260807 15:09:42.720615 19752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 489c497ec92f4d7bbc6541ff9a96ed36 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:42.720851 19753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 489c497ec92f4d7bbc6541ff9a96ed36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "489c497ec92f4d7bbc6541ff9a96ed36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "489c497ec92f4d7bbc6541ff9a96ed36" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34183 } } }
I20260807 15:09:42.721024 19753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 489c497ec92f4d7bbc6541ff9a96ed36 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:42.721140 19763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:09:42.725359 19763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:09:42.731984 19763 catalog_manager.cc:1347] Generated new cluster ID: ee33e92299ed45eda7f8402bbabe56f6
I20260807 15:09:42.732113 19763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:09:42.744853 19763 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:09:42.746285 19763 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:09:42.763846 19763 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 489c497ec92f4d7bbc6541ff9a96ed36: Generated new TSK 0
I20260807 15:09:42.764846 19763 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.226.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
I20260807 15:09:42.806946 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34183
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:40123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:42.930437 19784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:42.930794 19784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:42.930845 19784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:42.935451 19784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:42.935549 19784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:42.935617 19784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:09:42.941418 19784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:40123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:42.942561 19784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:42.943647 19784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:42.953890 19797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:42.953917 19793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:42.953974 19794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:42.954301 19784 server_base.cc:1034] running on GCE node
I20260807 15:09:42.954867 19784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:42.955648 19784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:42.956875 19784 hybrid_clock.cc:648] HybridClock initialized: now 1786115382956857 us; error 34 us; skew 500 ppm
Aug 07 15:09:42 dist-test-slave-3kk6 krb5kdc[19638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115382, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:42.960347 19784 init.cc:377] Logged in from keytab as kudu/127.15.226.65@KRBTEST.COM (short username kudu)
I20260807 15:09:42.961313 19784 webserver.cc:460] Webserver started at http://127.15.226.65:40241/ using document root <none> and password file <none>
I20260807 15:09:42.961963 19784 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:42.962065 19784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:42.962318 19784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:42.964677 19784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "af955108eca24094be7b8e52612deaff"
format_stamp: "Formatted at 2026-08-07 15:09:42 on dist-test-slave-3kk6"
server_key: "42aadedaed868634c15c6a0cd04d92df"
server_key_iv: "00c0b84fdc2c18dc65acebd39b9aa253"
server_key_version: "encryptionkey@0"
I20260807 15:09:42.965315 19784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "af955108eca24094be7b8e52612deaff"
format_stamp: "Formatted at 2026-08-07 15:09:42 on dist-test-slave-3kk6"
server_key: "42aadedaed868634c15c6a0cd04d92df"
server_key_iv: "00c0b84fdc2c18dc65acebd39b9aa253"
server_key_version: "encryptionkey@0"
I20260807 15:09:42.969480 19784 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:09:42.972285 19812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:42.973596 19784 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:09:42.973798 19784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "af955108eca24094be7b8e52612deaff"
format_stamp: "Formatted at 2026-08-07 15:09:42 on dist-test-slave-3kk6"
server_key: "42aadedaed868634c15c6a0cd04d92df"
server_key_iv: "00c0b84fdc2c18dc65acebd39b9aa253"
server_key_version: "encryptionkey@0"
I20260807 15:09:42.973939 19784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:42.987349 19784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:42.991025 19784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:42.991259 19784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:42.991941 19784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:42.992825 19784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:42.992899 19784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:42.992976 19784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:42.993021 19784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:43.002614 19784 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:38017
I20260807 15:09:43.002635 19965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:38017 every 8 connection(s)
I20260807 15:09:43.003721 19784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:09:43.005424 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 19784
I20260807 15:09:43.005522 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:09:43.005918 16265 external_mini_cluster.cc:1428] Setting key 6880f4f0c7acac1eeb764026fa67b8f5
Aug 07 15:09:43 dist-test-slave-3kk6 krb5kdc[19638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115382, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.226.66@KRBTEST.COM; defaulting to no policy
I20260807 15:09:43.019932 19749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:43.005784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:40537 (local address 127.15.226.126:34183)
0807 15:09:43.005965 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:43.005971 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:43.007083 (+ 1112us) server_negotiation.cc:407] Connection header received
0807 15:09:43.008963 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:43.008966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:43.009010 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:43.009083 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:43.010584 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:43.011241 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:43.012419 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:43.012722 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:43.012850 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:43.016013 (+ 3163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:43.016018 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:43.016021 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:43.016057 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:43.017817 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:43.018314 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:43.018317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:43.018318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:43.018395 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:43.018875 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:43.018877 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:43.018878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:43.019069 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:43.019150 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:43.019670 (+ 520us) server_negotiation.cc:299] Negotiation successful
0807 15:09:43.019799 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.66@KRBTEST.COM" created.
I20260807 15:09:43.021083 19971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:43.006291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34183 (local address 127.15.226.65:40537)
0807 15:09:43.006902 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:43.006961 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:43.008707 (+ 1746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:43.009307 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:43.009318 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:43.009876 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:43.010429 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:43.010444 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:43.011375 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:43.011378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:43.012285 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:43.012295 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:43.012897 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:43.012901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:43.013139 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:43.013940 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:43.013969 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:43.015814 (+ 1845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:43.017931 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:43.017936 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:43.017949 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:43.018219 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:43.018523 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:43.018529 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:43.018534 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:43.018775 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:43.019166 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:43.019171 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:43.019443 (+ 272us) client_negotiation.cc:769] Sending connection context
0807 15:09:43.019608 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 15:09:43.020061 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":160,"threads_started":1}
I20260807 15:09:43.022750 19966 heartbeater.cc:347] Connected to a master server at 127.15.226.126:34183
I20260807 15:09:43.023052 19966 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:43.023736 19966 heartbeater.cc:511] Master 127.15.226.126:34183 requested a full tablet report, sending...
I20260807 15:09:43.025635 19690 ts_manager.cc:194] Registered new tserver with Master: af955108eca24094be7b8e52612deaff (127.15.226.65:38017)
I20260807 15:09:43.027498 19690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.65@KRBTEST.COM'} at 127.15.226.65:40537
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
I20260807 15:09:43.039806 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34183
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:40123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:43.158442 19974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:43.158825 19974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:43.158877 19974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:43.163296 19974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:43.163371 19974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:43.163434 19974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:09:43.168987 19974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:40123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:43.170077 19974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:43.171177 19974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:43.181430 19983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:43.181430 19986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:43.181468 19984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:43.181758 19974 server_base.cc:1034] running on GCE node
I20260807 15:09:43.182296 19974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:43.183049 19974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:43.184216 19974 hybrid_clock.cc:648] HybridClock initialized: now 1786115383184196 us; error 32 us; skew 500 ppm
Aug 07 15:09:43 dist-test-slave-3kk6 krb5kdc[19638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115383, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:43.187204 19974 init.cc:377] Logged in from keytab as kudu/127.15.226.66@KRBTEST.COM (short username kudu)
I20260807 15:09:43.188194 19974 webserver.cc:460] Webserver started at http://127.15.226.66:39501/ using document root <none> and password file <none>
I20260807 15:09:43.188791 19974 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:43.188866 19974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:43.189141 19974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:43.191370 19974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "37198b7126a54f6c8f086c9acf7c3e41"
format_stamp: "Formatted at 2026-08-07 15:09:43 on dist-test-slave-3kk6"
server_key: "f947631df4c0e88025219e9a4f85a744"
server_key_iv: "7611207ae657daf84c40277ae89a58b5"
server_key_version: "encryptionkey@0"
I20260807 15:09:43.192011 19974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "37198b7126a54f6c8f086c9acf7c3e41"
format_stamp: "Formatted at 2026-08-07 15:09:43 on dist-test-slave-3kk6"
server_key: "f947631df4c0e88025219e9a4f85a744"
server_key_iv: "7611207ae657daf84c40277ae89a58b5"
server_key_version: "encryptionkey@0"
I20260807 15:09:43.195991 19974 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:09:43.198604 19995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:43.199928 19974 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:09:43.200084 19974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "37198b7126a54f6c8f086c9acf7c3e41"
format_stamp: "Formatted at 2026-08-07 15:09:43 on dist-test-slave-3kk6"
server_key: "f947631df4c0e88025219e9a4f85a744"
server_key_iv: "7611207ae657daf84c40277ae89a58b5"
server_key_version: "encryptionkey@0"
I20260807 15:09:43.200214 19974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:43.216790 19974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:43.220362 19974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:43.220592 19974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:43.221333 19974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:43.222286 19974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:43.222358 19974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:43.222429 19974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:43.222472 19974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:43.231878 19974 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:43487
I20260807 15:09:43.231900 20155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:43487 every 8 connection(s)
I20260807 15:09:43.232863 19974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:09:43.238456 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 19974
I20260807 15:09:43.238729 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:09:43.239068 16265 external_mini_cluster.cc:1428] Setting key d36d4937deeac2aa0f0bb4b065af8d6e
Aug 07 15:09:43 dist-test-slave-3kk6 krb5kdc[19638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115383, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:43.252884 19749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:43.234824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:52865 (local address 127.15.226.126:34183)
0807 15:09:43.234981 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:43.234985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:43.235802 (+ 817us) server_negotiation.cc:407] Connection header received
0807 15:09:43.237150 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:43.237153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:43.237216 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:43.237325 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:43.238573 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:43.239443 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:43.241198 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:43.241507 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:43.241631 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:43.245214 (+ 3583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:43.245218 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:43.245221 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:43.245253 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:43.249948 (+ 4695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:43.250787 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:43.250793 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:43.250798 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:43.250910 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:43.251406 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:43.251410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:43.251412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:43.251683 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:43.251795 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:43.252619 (+ 824us) server_negotiation.cc:299] Negotiation successful
0807 15:09:43.252745 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
WARNING: no policy specified for kudu/127.15.226.67@KRBTEST.COM; defaulting to no policy
I20260807 15:09:43.254204 20158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:43.235190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34183 (local address 127.15.226.66:52865)
0807 15:09:43.235644 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:43.235690 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:43.236931 (+ 1241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:43.237515 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:43.237523 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:43.237914 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:43.238415 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:43.238429 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:43.239622 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:43.239626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:43.241061 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:43.241071 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:43.241710 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:43.241716 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:43.241991 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:43.242821 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:43.242848 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:43.245001 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:43.250146 (+ 5145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:43.250157 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:43.250180 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:43.250631 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:43.251029 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:43.251034 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:43.251037 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:43.251263 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:43.251892 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:43.251901 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:43.252282 (+ 381us) client_negotiation.cc:769] Sending connection context
0807 15:09:43.252521 (+ 239us) client_negotiation.cc:240] Negotiation successful
0807 15:09:43.252961 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":111,"threads_started":1}
I20260807 15:09:43.255541 20156 heartbeater.cc:347] Connected to a master server at 127.15.226.126:34183
I20260807 15:09:43.255815 20156 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:43.256424 20156 heartbeater.cc:511] Master 127.15.226.126:34183 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.67@KRBTEST.COM" created.
I20260807 15:09:43.257682 19690 ts_manager.cc:194] Registered new tserver with Master: 37198b7126a54f6c8f086c9acf7c3e41 (127.15.226.66:43487)
I20260807 15:09:43.258339 19690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.66@KRBTEST.COM'} at 127.15.226.66:52865
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
I20260807 15:09:43.274737 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34183
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:40123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:43.393469 20161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:43.393782 20161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:43.393829 20161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:43.398033 20161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:43.398114 20161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:43.398228 20161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:09:43.403781 20161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:40123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:43.404994 20161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:43.406090 20161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:43.416143 20166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:43.416206 20161 server_base.cc:1034] running on GCE node
W20260807 15:09:43.416143 20170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:43.416143 20168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:43.416808 20161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:43.417569 20161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:43.418745 20161 hybrid_clock.cc:648] HybridClock initialized: now 1786115383418728 us; error 31 us; skew 500 ppm
Aug 07 15:09:43 dist-test-slave-3kk6 krb5kdc[19638](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115383, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:43.421865 20161 init.cc:377] Logged in from keytab as kudu/127.15.226.67@KRBTEST.COM (short username kudu)
I20260807 15:09:43.422921 20161 webserver.cc:460] Webserver started at http://127.15.226.67:33243/ using document root <none> and password file <none>
I20260807 15:09:43.423506 20161 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:43.423604 20161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:43.423857 20161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:43.426013 20161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "650426b6f07f45e9991be69bba1259ca"
format_stamp: "Formatted at 2026-08-07 15:09:43 on dist-test-slave-3kk6"
server_key: "1236899db0764d370b3032d3b6bd857c"
server_key_iv: "02067895001bd9a6aae1bb2185ccd2f9"
server_key_version: "encryptionkey@0"
I20260807 15:09:43.426645 20161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "650426b6f07f45e9991be69bba1259ca"
format_stamp: "Formatted at 2026-08-07 15:09:43 on dist-test-slave-3kk6"
server_key: "1236899db0764d370b3032d3b6bd857c"
server_key_iv: "02067895001bd9a6aae1bb2185ccd2f9"
server_key_version: "encryptionkey@0"
I20260807 15:09:43.430588 20161 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260807 15:09:43.433112 20181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:43.434293 20161 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 15:09:43.434451 20161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "650426b6f07f45e9991be69bba1259ca"
format_stamp: "Formatted at 2026-08-07 15:09:43 on dist-test-slave-3kk6"
server_key: "1236899db0764d370b3032d3b6bd857c"
server_key_iv: "02067895001bd9a6aae1bb2185ccd2f9"
server_key_version: "encryptionkey@0"
I20260807 15:09:43.434572 20161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:43.460036 20161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:43.463291 20161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:43.463521 20161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:43.464165 20161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:43.465106 20161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:43.465181 20161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:43.465260 20161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:43.465303 20161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:43.475163 20161 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:45641
I20260807 15:09:43.475186 20341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:45641 every 8 connection(s)
I20260807 15:09:43.476086 20161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:09:43.483346 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 20161
I20260807 15:09:43.483454 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:09:43.483771 16265 external_mini_cluster.cc:1428] Setting key 381ca3b79a5c671d211a18f99c97af56
Aug 07 15:09:43 dist-test-slave-3kk6 krb5kdc[19638](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115383, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:43.490779 19749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:43.478099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:37395 (local address 127.15.226.126:34183)
0807 15:09:43.478227 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:43.478231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:43.479159 (+ 928us) server_negotiation.cc:407] Connection header received
0807 15:09:43.480533 (+ 1374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:43.480536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:43.480586 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:43.480665 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:43.481812 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:43.482349 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:43.483902 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:43.484148 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:43.484294 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:43.487018 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:43.487022 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:43.487025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:43.487052 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:43.488578 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:43.489145 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:43.489149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:43.489150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:43.489230 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:43.489599 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:43.489603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:43.489605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:43.489803 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:43.489914 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:43.490447 (+ 533us) server_negotiation.cc:299] Negotiation successful
0807 15:09:43.490576 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 15:09:43.491638 20344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:43.478487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34183 (local address 127.15.226.67:37395)
0807 15:09:43.478982 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:43.479026 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:43.480320 (+ 1294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:43.480825 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:43.480831 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:43.481190 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:43.481676 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:43.481689 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:43.482501 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:43.482504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:43.483711 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:43.483724 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:43.484315 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:43.484319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:43.484532 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:43.485247 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:43.485264 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:43.486850 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:43.488710 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:43.488716 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:43.488733 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:43.489020 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:43.489333 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:43.489336 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:43.489339 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:43.489492 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:43.489914 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:43.489923 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:43.490193 (+ 270us) client_negotiation.cc:769] Sending connection context
0807 15:09:43.490386 (+ 193us) client_negotiation.cc:240] Negotiation successful
0807 15:09:43.490781 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":172,"threads_started":1}
I20260807 15:09:43.492924 20342 heartbeater.cc:347] Connected to a master server at 127.15.226.126:34183
I20260807 15:09:43.493160 20342 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:43.493638 20342 heartbeater.cc:511] Master 127.15.226.126:34183 requested a full tablet report, sending...
I20260807 15:09:43.494784 19690 ts_manager.cc:194] Registered new tserver with Master: 650426b6f07f45e9991be69bba1259ca (127.15.226.67:45641)
I20260807 15:09:43.495380 19690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.67@KRBTEST.COM'} at 127.15.226.67:37395
I20260807 15:09:43.497743 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 15:09:43.513432 19749 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:09:43.511908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36318 (local address 127.15.226.126:34183)
0807 15:09:43.512125 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:43.512129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:43.512198 (+ 69us) server_negotiation.cc:407] Connection header received
0807 15:09:43.512332 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:43.512335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:43.512382 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:43.512461 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:43.513267 (+ 806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36318: BlockingRecv error: recv got EOF from 127.0.0.1:36318 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 15:09:43.514544 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 19784
I20260807 15:09:43.534178 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 19974
I20260807 15:09:43.549793 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 20161
I20260807 15:09:43.563750 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 19655
2026-08-07T15:09:43Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2480 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:09:43 dist-test-slave-3kk6 krb5kdc[20357](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:09:43 dist-test-slave-3kk6 krb5kdc[20357](info): set up 2 sockets
Aug 07 15:09:43 dist-test-slave-3kk6 krb5kdc[20357](info): commencing operation
krb5kdc: starting...
W20260807 15:09:44.820923 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:09:44 dist-test-slave-3kk6 krb5kdc[20357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:09:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:09:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:09:44.963172 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:37947
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:40315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:37947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:45.082000 20375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:45.082386 20375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:45.082472 20375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:45.086892 20375 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:45.087023 20375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:45.087059 20375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:45.087103 20375 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:45.087141 20375 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:09:45.092962 20375 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:40315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:37947
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:37947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:45.094256 20375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:45.095515 20375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:45.104771 20388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:45.104830 20386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:45.104776 20385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:45.105039 20375 server_base.cc:1034] running on GCE node
I20260807 15:09:45.105731 20375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:45.107007 20375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:45.108227 20375 hybrid_clock.cc:648] HybridClock initialized: now 1786115385108196 us; error 48 us; skew 500 ppm
Aug 07 15:09:45 dist-test-slave-3kk6 krb5kdc[20357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115385, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:45.111414 20375 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:09:45.112468 20375 webserver.cc:460] Webserver started at http://127.15.226.126:33993/ using document root <none> and password file <none>
I20260807 15:09:45.113078 20375 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:45.113183 20375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:45.113446 20375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:45.115800 20375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "465891b0ddd2415daede5bbb37c8e011"
format_stamp: "Formatted at 2026-08-07 15:09:45 on dist-test-slave-3kk6"
server_key: "52a0e53da43194c987710a7898eb2b62"
server_key_iv: "c732bf1367b74d076e905e2628b560c2"
server_key_version: "encryptionkey@0"
I20260807 15:09:45.116454 20375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "465891b0ddd2415daede5bbb37c8e011"
format_stamp: "Formatted at 2026-08-07 15:09:45 on dist-test-slave-3kk6"
server_key: "52a0e53da43194c987710a7898eb2b62"
server_key_iv: "c732bf1367b74d076e905e2628b560c2"
server_key_version: "encryptionkey@0"
I20260807 15:09:45.120651 20375 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:09:45.123487 20400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:45.124827 20375 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 15:09:45.125027 20375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "465891b0ddd2415daede5bbb37c8e011"
format_stamp: "Formatted at 2026-08-07 15:09:45 on dist-test-slave-3kk6"
server_key: "52a0e53da43194c987710a7898eb2b62"
server_key_iv: "c732bf1367b74d076e905e2628b560c2"
server_key_version: "encryptionkey@0"
I20260807 15:09:45.125180 20375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:45.159425 20375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:45.163057 20375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:45.163309 20375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:45.171118 20375 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:37947
I20260807 15:09:45.171502 20467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:37947 every 8 connection(s)
I20260807 15:09:45.172341 20375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:09:45.175508 20470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:45.181481 20470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 465891b0ddd2415daede5bbb37c8e011: Bootstrap starting.
I20260807 15:09:45.182168 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 20375
I20260807 15:09:45.182337 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:09:45.182633 16265 external_mini_cluster.cc:1428] Setting key 788acf178e1bbee3ad5b2052b2c10148
I20260807 15:09:45.184536 20470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 465891b0ddd2415daede5bbb37c8e011: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:45.185351 20470 log.cc:824] T 00000000000000000000000000000000 P 465891b0ddd2415daede5bbb37c8e011: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:45.187803 20470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 465891b0ddd2415daede5bbb37c8e011: No bootstrap required, opened a new log
Aug 07 15:09:45 dist-test-slave-3kk6 krb5kdc[20357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:45.192826 20470 raft_consensus.cc:348] T 00000000000000000000000000000000 P 465891b0ddd2415daede5bbb37c8e011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "465891b0ddd2415daede5bbb37c8e011" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37947 } }
I20260807 15:09:45.193101 20470 raft_consensus.cc:374] T 00000000000000000000000000000000 P 465891b0ddd2415daede5bbb37c8e011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:45.193199 20470 raft_consensus.cc:729] T 00000000000000000000000000000000 P 465891b0ddd2415daede5bbb37c8e011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 465891b0ddd2415daede5bbb37c8e011, State: Initialized, Role: FOLLOWER
I20260807 15:09:45.193713 20470 consensus_queue.cc:260] T 00000000000000000000000000000000 P 465891b0ddd2415daede5bbb37c8e011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "465891b0ddd2415daede5bbb37c8e011" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37947 } }
I20260807 15:09:45.193892 20470 raft_consensus.cc:388] T 00000000000000000000000000000000 P 465891b0ddd2415daede5bbb37c8e011 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:09:45.193980 20470 raft_consensus.cc:482] T 00000000000000000000000000000000 P 465891b0ddd2415daede5bbb37c8e011 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:09:45.194115 20470 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 465891b0ddd2415daede5bbb37c8e011 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:45.195385 20470 raft_consensus.cc:504] T 00000000000000000000000000000000 P 465891b0ddd2415daede5bbb37c8e011 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "465891b0ddd2415daede5bbb37c8e011" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37947 } }
I20260807 15:09:45.195761 20470 leader_election.cc:302] T 00000000000000000000000000000000 P 465891b0ddd2415daede5bbb37c8e011 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 465891b0ddd2415daede5bbb37c8e011; no voters:
I20260807 15:09:45.196148 20470 leader_election.cc:288] T 00000000000000000000000000000000 P 465891b0ddd2415daede5bbb37c8e011 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:09:45.196242 20477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 465891b0ddd2415daede5bbb37c8e011 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:45.196511 20477 raft_consensus.cc:686] T 00000000000000000000000000000000 P 465891b0ddd2415daede5bbb37c8e011 [term 1 LEADER]: Becoming Leader. State: Replica: 465891b0ddd2415daede5bbb37c8e011, State: Running, Role: LEADER
I20260807 15:09:45.196898 20477 consensus_queue.cc:237] T 00000000000000000000000000000000 P 465891b0ddd2415daede5bbb37c8e011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "465891b0ddd2415daede5bbb37c8e011" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37947 } }
I20260807 15:09:45.197620 20470 sys_catalog.cc:564] T 00000000000000000000000000000000 P 465891b0ddd2415daede5bbb37c8e011 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:45.198769 20478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 465891b0ddd2415daede5bbb37c8e011 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "465891b0ddd2415daede5bbb37c8e011" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "465891b0ddd2415daede5bbb37c8e011" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37947 } } }
I20260807 15:09:45.198913 20478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 465891b0ddd2415daede5bbb37c8e011 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:45.199357 20479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 465891b0ddd2415daede5bbb37c8e011 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 465891b0ddd2415daede5bbb37c8e011. Latest consensus state: current_term: 1 leader_uuid: "465891b0ddd2415daede5bbb37c8e011" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "465891b0ddd2415daede5bbb37c8e011" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37947 } } }
I20260807 15:09:45.199504 20479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 465891b0ddd2415daede5bbb37c8e011 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:45.199834 20490 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:09:45.200338 20475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:45.184051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35380 (local address 127.15.226.126:37947)
0807 15:09:45.184574 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:45.184621 (+ 47us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:45.184672 (+ 51us) server_negotiation.cc:407] Connection header received
0807 15:09:45.185379 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:45.185398 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:45.185870 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:45.186287 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:45.187372 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:45.188505 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:45.189746 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:45.190289 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:45.190434 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:45.193792 (+ 3358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:45.193797 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:45.193840 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:45.193886 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:45.196208 (+ 2322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:45.196890 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:45.196898 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:45.196905 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:45.197053 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:45.197391 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:45.197394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:45.197396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:45.197914 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:45.198148 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:45.198902 (+ 754us) server_negotiation.cc:299] Negotiation successful
0807 15:09:45.199258 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":180,"threads_started":1}
I20260807 15:09:45.203466 20490 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:09:45.204041 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:09:45.210903 20490 catalog_manager.cc:1347] Generated new cluster ID: 870e281ad408489191d2b5e23f18c28e
I20260807 15:09:45.211016 20490 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:09:45.227176 20475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:45.220845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35384 (local address 127.15.226.126:37947)
0807 15:09:45.221006 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:45.221012 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:45.221309 (+ 297us) server_negotiation.cc:407] Connection header received
0807 15:09:45.221458 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:45.221470 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:45.221528 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:45.221616 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:45.222316 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:45.223149 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:45.224406 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:45.224758 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:45.224855 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:45.225900 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:45.225904 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:45.225907 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:09:45.226624 (+ 717us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:45.226703 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:09:45.226724 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:45.226809 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:45.226944 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 15:09:45.227020 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 15:09:45.238808 20490 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:09:45.239881 20490 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:09:45.257666 20490 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 465891b0ddd2415daede5bbb37c8e011: Generated new TSK 0
I20260807 15:09:45.258492 20490 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:09:45.266530 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 20375
2026-08-07T15:09:45Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1699 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:09:45 dist-test-slave-3kk6 krb5kdc[20513](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:09:45 dist-test-slave-3kk6 krb5kdc[20513](info): set up 2 sockets
Aug 07 15:09:45 dist-test-slave-3kk6 krb5kdc[20513](info): commencing operation
krb5kdc: starting...
W20260807 15:09:46.518333 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:09:46 dist-test-slave-3kk6 krb5kdc[20513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:09:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:09:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:09:46.658821 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:41057
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:35881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:41057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:46.780058 20538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:46.780439 20538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:46.780521 20538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:46.784763 20538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:46.784878 20538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:46.784917 20538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:46.784958 20538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:46.784996 20538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:09:46.790926 20538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:35881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:41057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:41057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:46.792227 20538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:46.793387 20538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:46.802803 20543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:46.802803 20544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:46.802824 20547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:46.803188 20538 server_base.cc:1034] running on GCE node
I20260807 15:09:46.803849 20538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:46.805058 20538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:46.806289 20538 hybrid_clock.cc:648] HybridClock initialized: now 1786115386806262 us; error 51 us; skew 500 ppm
Aug 07 15:09:46 dist-test-slave-3kk6 krb5kdc[20513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115386, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:46.809583 20538 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:09:46.810619 20538 webserver.cc:460] Webserver started at http://127.15.226.126:44705/ using document root <none> and password file <none>
I20260807 15:09:46.811256 20538 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:46.811322 20538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:46.811504 20538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:46.813815 20538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "cbd9c88bbe8f4ddebcfba0b8da3152c8"
format_stamp: "Formatted at 2026-08-07 15:09:46 on dist-test-slave-3kk6"
server_key: "6b4c1614524774a256124ad9eb8b6a31"
server_key_iv: "d9a9af81c7737ab37e74dba7e6adc267"
server_key_version: "encryptionkey@0"
I20260807 15:09:46.814436 20538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "cbd9c88bbe8f4ddebcfba0b8da3152c8"
format_stamp: "Formatted at 2026-08-07 15:09:46 on dist-test-slave-3kk6"
server_key: "6b4c1614524774a256124ad9eb8b6a31"
server_key_iv: "d9a9af81c7737ab37e74dba7e6adc267"
server_key_version: "encryptionkey@0"
I20260807 15:09:46.818578 20538 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 15:09:46.821285 20555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:46.822629 20538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 15:09:46.822844 20538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "cbd9c88bbe8f4ddebcfba0b8da3152c8"
format_stamp: "Formatted at 2026-08-07 15:09:46 on dist-test-slave-3kk6"
server_key: "6b4c1614524774a256124ad9eb8b6a31"
server_key_iv: "d9a9af81c7737ab37e74dba7e6adc267"
server_key_version: "encryptionkey@0"
I20260807 15:09:46.822962 20538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:46.840870 20538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:46.844787 20538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:46.845028 20538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:46.852691 20538 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:41057
I20260807 15:09:46.853128 20635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:41057 every 8 connection(s)
I20260807 15:09:46.853966 20538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:09:46.857086 20636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:46.857766 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 20538
I20260807 15:09:46.857924 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:09:46.858222 16265 external_mini_cluster.cc:1428] Setting key 41663c3e786d5e887c3860f3c1a1401b
I20260807 15:09:46.864598 20636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbd9c88bbe8f4ddebcfba0b8da3152c8: Bootstrap starting.
I20260807 15:09:46.867677 20636 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbd9c88bbe8f4ddebcfba0b8da3152c8: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:46.868623 20636 log.cc:824] T 00000000000000000000000000000000 P cbd9c88bbe8f4ddebcfba0b8da3152c8: Log is configured to *not* fsync() on all Append() calls
Aug 07 15:09:46 dist-test-slave-3kk6 krb5kdc[20513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:46.871320 20636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbd9c88bbe8f4ddebcfba0b8da3152c8: No bootstrap required, opened a new log
I20260807 15:09:46.875186 20636 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbd9c88bbe8f4ddebcfba0b8da3152c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbd9c88bbe8f4ddebcfba0b8da3152c8" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 41057 } }
I20260807 15:09:46.875489 20636 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbd9c88bbe8f4ddebcfba0b8da3152c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:46.875583 20639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:46.859794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56354 (local address 127.15.226.126:41057)
0807 15:09:46.860311 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:46.860338 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:46.860386 (+ 48us) server_negotiation.cc:407] Connection header received
0807 15:09:46.861262 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:46.861289 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:46.861749 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:46.862145 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:46.863057 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:46.864594 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:46.865707 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:46.866473 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:46.866690 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:46.870161 (+ 3471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:46.870167 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:46.870184 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:46.870222 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:46.872265 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:46.872817 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:46.872822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:46.872828 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:46.872968 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:46.873285 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:46.873289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:46.873291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:46.873746 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:46.873962 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:46.874308 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 15:09:46.874706 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":153,"threads_started":1}
I20260807 15:09:46.875592 20636 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbd9c88bbe8f4ddebcfba0b8da3152c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbd9c88bbe8f4ddebcfba0b8da3152c8, State: Initialized, Role: FOLLOWER
I20260807 15:09:46.876206 20636 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbd9c88bbe8f4ddebcfba0b8da3152c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbd9c88bbe8f4ddebcfba0b8da3152c8" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 41057 } }
I20260807 15:09:46.876397 20636 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbd9c88bbe8f4ddebcfba0b8da3152c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:09:46.876478 20636 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbd9c88bbe8f4ddebcfba0b8da3152c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:09:46.876580 20636 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbd9c88bbe8f4ddebcfba0b8da3152c8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:46.877651 20636 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbd9c88bbe8f4ddebcfba0b8da3152c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbd9c88bbe8f4ddebcfba0b8da3152c8" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 41057 } }
I20260807 15:09:46.878036 20636 leader_election.cc:302] T 00000000000000000000000000000000 P cbd9c88bbe8f4ddebcfba0b8da3152c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbd9c88bbe8f4ddebcfba0b8da3152c8; no voters:
I20260807 15:09:46.878403 20636 leader_election.cc:288] T 00000000000000000000000000000000 P cbd9c88bbe8f4ddebcfba0b8da3152c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:09:46.878630 20641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbd9c88bbe8f4ddebcfba0b8da3152c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:46.879053 20641 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbd9c88bbe8f4ddebcfba0b8da3152c8 [term 1 LEADER]: Becoming Leader. State: Replica: cbd9c88bbe8f4ddebcfba0b8da3152c8, State: Running, Role: LEADER
I20260807 15:09:46.879420 20641 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbd9c88bbe8f4ddebcfba0b8da3152c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbd9c88bbe8f4ddebcfba0b8da3152c8" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 41057 } }
I20260807 15:09:46.879809 20636 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbd9c88bbe8f4ddebcfba0b8da3152c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:46.881343 20642 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbd9c88bbe8f4ddebcfba0b8da3152c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cbd9c88bbe8f4ddebcfba0b8da3152c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbd9c88bbe8f4ddebcfba0b8da3152c8" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 41057 } } }
I20260807 15:09:46.881511 20642 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbd9c88bbe8f4ddebcfba0b8da3152c8 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:46.881973 20652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:09:46.881914 20643 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbd9c88bbe8f4ddebcfba0b8da3152c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbd9c88bbe8f4ddebcfba0b8da3152c8. Latest consensus state: current_term: 1 leader_uuid: "cbd9c88bbe8f4ddebcfba0b8da3152c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbd9c88bbe8f4ddebcfba0b8da3152c8" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 41057 } } }
I20260807 15:09:46.882082 20643 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbd9c88bbe8f4ddebcfba0b8da3152c8 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:46.885195 20652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:09:46.891115 20652 catalog_manager.cc:1347] Generated new cluster ID: 29008ead5dd1400d8474cc49d9b8e634
I20260807 15:09:46.891220 20652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:09:46.904209 20652 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:09:46.905293 20652 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:09:46.912458 20652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbd9c88bbe8f4ddebcfba0b8da3152c8: Generated new TSK 0
I20260807 15:09:46.913329 20652 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.226.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
I20260807 15:09:46.952346 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:41057
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:35881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:47.078544 20668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:47.078919 20668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:47.078969 20668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:47.083606 20668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:47.083699 20668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:47.083765 20668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:09:47.089516 20668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:35881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:41057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:47.090798 20668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:47.091876 20668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:47.103071 20675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:47.103073 20676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:47.103158 20668 server_base.cc:1034] running on GCE node
W20260807 15:09:47.103179 20679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:47.103796 20668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:47.104498 20668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:47.105718 20668 hybrid_clock.cc:648] HybridClock initialized: now 1786115387105705 us; error 33 us; skew 500 ppm
Aug 07 15:09:47 dist-test-slave-3kk6 krb5kdc[20513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115387, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:47.108969 20668 init.cc:377] Logged in from keytab as kudu/127.15.226.65@KRBTEST.COM (short username kudu)
I20260807 15:09:47.110046 20668 webserver.cc:460] Webserver started at http://127.15.226.65:35275/ using document root <none> and password file <none>
I20260807 15:09:47.110771 20668 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:47.110869 20668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:47.111109 20668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:47.113319 20668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "f672c64c39a641c9a38405933410c9e8"
format_stamp: "Formatted at 2026-08-07 15:09:47 on dist-test-slave-3kk6"
server_key: "51dbd51d956a8c04f2a8590a2454c88f"
server_key_iv: "ee927d686170fcde5d1025d97efb0036"
server_key_version: "encryptionkey@0"
I20260807 15:09:47.113973 20668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "f672c64c39a641c9a38405933410c9e8"
format_stamp: "Formatted at 2026-08-07 15:09:47 on dist-test-slave-3kk6"
server_key: "51dbd51d956a8c04f2a8590a2454c88f"
server_key_iv: "ee927d686170fcde5d1025d97efb0036"
server_key_version: "encryptionkey@0"
I20260807 15:09:47.118180 20668 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:09:47.121172 20687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:47.122810 20668 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 15:09:47.122968 20668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "f672c64c39a641c9a38405933410c9e8"
format_stamp: "Formatted at 2026-08-07 15:09:47 on dist-test-slave-3kk6"
server_key: "51dbd51d956a8c04f2a8590a2454c88f"
server_key_iv: "ee927d686170fcde5d1025d97efb0036"
server_key_version: "encryptionkey@0"
I20260807 15:09:47.123108 20668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:47.169339 20668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:47.173491 20668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:47.173760 20668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:47.174484 20668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:47.175474 20668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:47.175552 20668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:47.175637 20668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:47.175683 20668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:47.186446 20668 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:35287
I20260807 15:09:47.186483 20843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:35287 every 8 connection(s)
I20260807 15:09:47.187557 20668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:09:47.192073 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 20668
I20260807 15:09:47.192199 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:09:47.192488 16265 external_mini_cluster.cc:1428] Setting key 7bf1ff37bf40a62ed88273200e7ee2a5
Aug 07 15:09:47 dist-test-slave-3kk6 krb5kdc[20513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115387, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.226.66@KRBTEST.COM; defaulting to no policy
I20260807 15:09:47.204725 20639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:47.189695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:57317 (local address 127.15.226.126:41057)
0807 15:09:47.189888 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:47.189893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:47.190917 (+ 1024us) server_negotiation.cc:407] Connection header received
0807 15:09:47.192761 (+ 1844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:47.192765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:47.192831 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:47.192947 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:47.194527 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:47.195178 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:47.196502 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:47.196773 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:47.196870 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:47.200444 (+ 3574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:47.200448 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:47.200451 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:47.200480 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:47.202181 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:47.202878 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:47.202882 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:47.202886 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:47.202977 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:47.203381 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:47.203385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:47.203388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:47.203600 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:47.203725 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:47.204388 (+ 663us) server_negotiation.cc:299] Negotiation successful
0807 15:09:47.204583 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 15:09:47.205611 20847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:47.190120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:41057 (local address 127.15.226.65:57317)
0807 15:09:47.190680 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:47.190778 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:47.192459 (+ 1681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:47.193152 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:47.193160 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:47.193728 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:47.194365 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:47.194381 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:47.195320 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:47.195324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:47.196333 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:47.196349 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:47.196963 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:47.196969 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:47.197223 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:47.198004 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:47.198033 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:47.200194 (+ 2161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:47.202337 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:47.202345 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:47.202373 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:47.202750 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:47.203083 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:47.203086 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:47.203088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:47.203265 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:47.203718 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:47.203726 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:47.204031 (+ 305us) client_negotiation.cc:769] Sending connection context
0807 15:09:47.204261 (+ 230us) client_negotiation.cc:240] Negotiation successful
0807 15:09:47.204605 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":135,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.66@KRBTEST.COM" created.
I20260807 15:09:47.207255 20845 heartbeater.cc:347] Connected to a master server at 127.15.226.126:41057
I20260807 15:09:47.207557 20845 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:47.208295 20845 heartbeater.cc:511] Master 127.15.226.126:41057 requested a full tablet report, sending...
I20260807 15:09:47.210165 20583 ts_manager.cc:194] Registered new tserver with Master: f672c64c39a641c9a38405933410c9e8 (127.15.226.65:35287)
I20260807 15:09:47.211895 20583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.65@KRBTEST.COM'} at 127.15.226.65:57317
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
I20260807 15:09:47.226150 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:41057
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:35881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:47.349763 20850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:47.350090 20850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:47.350136 20850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:47.354470 20850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:47.354554 20850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:47.354620 20850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:09:47.360363 20850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:35881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:41057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:47.361519 20850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:47.362689 20850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:47.374178 20861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:47.374167 20855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:47.374174 20856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:47.374552 20850 server_base.cc:1034] running on GCE node
I20260807 15:09:47.375195 20850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:47.375898 20850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:47.377104 20850 hybrid_clock.cc:648] HybridClock initialized: now 1786115387377098 us; error 36 us; skew 500 ppm
Aug 07 15:09:47 dist-test-slave-3kk6 krb5kdc[20513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115387, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:47.380369 20850 init.cc:377] Logged in from keytab as kudu/127.15.226.66@KRBTEST.COM (short username kudu)
I20260807 15:09:47.381603 20850 webserver.cc:460] Webserver started at http://127.15.226.66:42181/ using document root <none> and password file <none>
I20260807 15:09:47.382226 20850 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:47.382330 20850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:47.382593 20850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:47.384932 20850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "109626da413942f7a20670197dbda7dc"
format_stamp: "Formatted at 2026-08-07 15:09:47 on dist-test-slave-3kk6"
server_key: "f355abe31eb1a4f626f4012daa04ae74"
server_key_iv: "370c13e0f4b9801ca0d959eb0ae00285"
server_key_version: "encryptionkey@0"
I20260807 15:09:47.385632 20850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "109626da413942f7a20670197dbda7dc"
format_stamp: "Formatted at 2026-08-07 15:09:47 on dist-test-slave-3kk6"
server_key: "f355abe31eb1a4f626f4012daa04ae74"
server_key_iv: "370c13e0f4b9801ca0d959eb0ae00285"
server_key_version: "encryptionkey@0"
I20260807 15:09:47.390025 20850 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:09:47.393079 20871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:47.394608 20850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:09:47.394817 20850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "109626da413942f7a20670197dbda7dc"
format_stamp: "Formatted at 2026-08-07 15:09:47 on dist-test-slave-3kk6"
server_key: "f355abe31eb1a4f626f4012daa04ae74"
server_key_iv: "370c13e0f4b9801ca0d959eb0ae00285"
server_key_version: "encryptionkey@0"
I20260807 15:09:47.394968 20850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:47.410202 20850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:47.413985 20850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:47.414232 20850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:47.414976 20850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:47.415917 20850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:47.416004 20850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:47.416080 20850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:47.416126 20850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:47.425876 20850 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:46505
I20260807 15:09:47.425909 21025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:46505 every 8 connection(s)
I20260807 15:09:47.427098 20850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:09:47.435645 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 20850
I20260807 15:09:47.435768 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:09:47.436112 16265 external_mini_cluster.cc:1428] Setting key d97f81c9349b8edc0cde2b07802e845e
Aug 07 15:09:47 dist-test-slave-3kk6 krb5kdc[20513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115387, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:47.445012 20639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:47.429167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:46573 (local address 127.15.226.126:41057)
0807 15:09:47.429330 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:47.429336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:47.430337 (+ 1001us) server_negotiation.cc:407] Connection header received
0807 15:09:47.431889 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:47.431892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:47.431948 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:47.432032 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:47.433443 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:47.434035 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:47.435559 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:47.435887 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:47.435986 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:47.440560 (+ 4574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:47.440564 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:47.440566 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:47.440595 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:47.442285 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:47.443079 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:47.443082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:47.443084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:47.443210 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:47.443614 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:47.443618 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:47.443619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:47.443841 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:47.443954 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:47.444734 (+ 780us) server_negotiation.cc:299] Negotiation successful
0807 15:09:47.444865 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 15:09:47.446451 21028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:47.429641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:41057 (local address 127.15.226.66:46573)
0807 15:09:47.430151 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:47.430201 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:47.431626 (+ 1425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:47.432219 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:47.432226 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:47.432645 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:47.433253 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:47.433270 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:47.434184 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:47.434187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:47.435397 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:47.435414 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:47.436079 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:47.436083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:47.436323 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:47.437130 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:47.437152 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:47.439499 (+ 2347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:47.442474 (+ 2975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:47.442482 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:47.442510 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:47.442940 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:47.443309 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:47.443312 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:47.443314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:47.443499 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:47.443956 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:47.443966 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:47.444385 (+ 419us) client_negotiation.cc:769] Sending connection context
0807 15:09:47.444645 (+ 260us) client_negotiation.cc:240] Negotiation successful
0807 15:09:47.445134 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":125,"threads_started":1}
I20260807 15:09:47.447904 21026 heartbeater.cc:347] Connected to a master server at 127.15.226.126:41057
I20260807 15:09:47.448215 21026 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.15.226.67@KRBTEST.COM; defaulting to no policy
I20260807 15:09:47.448930 21026 heartbeater.cc:511] Master 127.15.226.126:41057 requested a full tablet report, sending...
I20260807 15:09:47.450341 20583 ts_manager.cc:194] Registered new tserver with Master: 109626da413942f7a20670197dbda7dc (127.15.226.66:46505)
I20260807 15:09:47.451252 20583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.66@KRBTEST.COM'} at 127.15.226.66:46573
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
I20260807 15:09:47.471138 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:41057
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:35881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:47.592235 21031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:47.592614 21031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:47.592701 21031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:47.597440 21031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:47.597579 21031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:47.597682 21031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:09:47.603746 21031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:35881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:41057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:47.605028 21031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:47.606172 21031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:47.616647 21039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:47.616647 21043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:47.616650 21041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:47.617141 21031 server_base.cc:1034] running on GCE node
I20260807 15:09:47.617604 21031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:47.618387 21031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:47.619593 21031 hybrid_clock.cc:648] HybridClock initialized: now 1786115387619560 us; error 50 us; skew 500 ppm
Aug 07 15:09:47 dist-test-slave-3kk6 krb5kdc[20513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115387, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:47.622812 21031 init.cc:377] Logged in from keytab as kudu/127.15.226.67@KRBTEST.COM (short username kudu)
I20260807 15:09:47.623767 21031 webserver.cc:460] Webserver started at http://127.15.226.67:39417/ using document root <none> and password file <none>
I20260807 15:09:47.624334 21031 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:47.624400 21031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:47.624600 21031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:47.626935 21031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "1188068508e043c38e43cc9b49da34c7"
format_stamp: "Formatted at 2026-08-07 15:09:47 on dist-test-slave-3kk6"
server_key: "1597cc791e7cda390a8696fcf1755a43"
server_key_iv: "d96a18e1654401fe2bbebe722b17835f"
server_key_version: "encryptionkey@0"
I20260807 15:09:47.627575 21031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "1188068508e043c38e43cc9b49da34c7"
format_stamp: "Formatted at 2026-08-07 15:09:47 on dist-test-slave-3kk6"
server_key: "1597cc791e7cda390a8696fcf1755a43"
server_key_iv: "d96a18e1654401fe2bbebe722b17835f"
server_key_version: "encryptionkey@0"
I20260807 15:09:47.631946 21031 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:09:47.634781 21053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:47.636101 21031 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:09:47.636253 21031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "1188068508e043c38e43cc9b49da34c7"
format_stamp: "Formatted at 2026-08-07 15:09:47 on dist-test-slave-3kk6"
server_key: "1597cc791e7cda390a8696fcf1755a43"
server_key_iv: "d96a18e1654401fe2bbebe722b17835f"
server_key_version: "encryptionkey@0"
I20260807 15:09:47.636394 21031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:47.651770 21031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:47.655939 21031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:47.656224 21031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:47.656951 21031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:47.657925 21031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:47.658002 21031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:47.658079 21031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:47.658125 21031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:47.668382 21031 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:36291
I20260807 15:09:47.668401 21219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:36291 every 8 connection(s)
I20260807 15:09:47.669442 21031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:09:47.669713 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 21031
I20260807 15:09:47.669843 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:09:47.670185 16265 external_mini_cluster.cc:1428] Setting key 3fbde6533456f01320acbcd6db5f7069
Aug 07 15:09:47 dist-test-slave-3kk6 krb5kdc[20513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115387, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:47.686522 20639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:47.671725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:36825 (local address 127.15.226.126:41057)
0807 15:09:47.671870 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:47.671874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:47.672935 (+ 1061us) server_negotiation.cc:407] Connection header received
0807 15:09:47.674393 (+ 1458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:47.674395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:47.674446 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:47.674523 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:47.676035 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:47.676602 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:47.677948 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:47.678291 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:47.678393 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:47.682068 (+ 3675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:47.682079 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:47.682081 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:47.682116 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:47.684057 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:47.684810 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:47.684813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:47.684815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:47.684914 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:47.685315 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:47.685318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:47.685320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:47.685561 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:47.685657 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:47.686236 (+ 579us) server_negotiation.cc:299] Negotiation successful
0807 15:09:47.686393 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:09:47.687542 21223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:47.672220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:41057 (local address 127.15.226.67:36825)
0807 15:09:47.672760 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:47.672829 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:47.674180 (+ 1351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:47.674848 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:47.674856 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:47.675296 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:47.675827 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:47.675843 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:47.676757 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:47.676768 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:47.677789 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:47.677802 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:47.678501 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:47.678504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:47.678814 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:47.679821 (+ 1007us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:47.679843 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:47.681839 (+ 1996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:47.684188 (+ 2349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:47.684196 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:47.684217 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:47.684678 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:47.685000 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:47.685003 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:47.685004 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:47.685176 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:47.685672 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:47.685678 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:47.685966 (+ 288us) client_negotiation.cc:769] Sending connection context
0807 15:09:47.686145 (+ 179us) client_negotiation.cc:240] Negotiation successful
0807 15:09:47.686465 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":158,"threads_started":1}
I20260807 15:09:47.688956 21221 heartbeater.cc:347] Connected to a master server at 127.15.226.126:41057
I20260807 15:09:47.689249 21221 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:47.689834 21221 heartbeater.cc:511] Master 127.15.226.126:41057 requested a full tablet report, sending...
I20260807 15:09:47.691071 20583 ts_manager.cc:194] Registered new tserver with Master: 1188068508e043c38e43cc9b49da34c7 (127.15.226.67:36291)
I20260807 15:09:47.691718 20583 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.67@KRBTEST.COM'} at 127.15.226.67:36825
I20260807 15:09:47.696936 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 15:09:47 dist-test-slave-3kk6 krb5kdc[20513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115387, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 15:09:47 dist-test-slave-3kk6 krb5kdc[20513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115387, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:47.727559 20639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:47.717429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56360 (local address 127.15.226.126:41057)
0807 15:09:47.717639 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:47.717644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:47.717728 (+ 84us) server_negotiation.cc:407] Connection header received
0807 15:09:47.717861 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:47.717864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:47.717914 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:47.717993 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:47.718814 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:47.719397 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:47.720274 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:47.720498 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:47.720592 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:47.723602 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:47.723606 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:47.723608 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:47.723642 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:47.725404 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:47.725992 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:47.725995 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:47.725997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:47.726095 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:47.726373 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:47.726376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:47.726377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:47.726744 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:47.726904 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:47.727195 (+ 291us) server_negotiation.cc:299] Negotiation successful
0807 15:09:47.727314 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
W20260807 15:09:47.728173 20583 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:56360
I20260807 15:09:47.729262 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 20668
I20260807 15:09:47.748771 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 20850
I20260807 15:09:47.768836 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 21031
I20260807 15:09:47.785519 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 20538
2026-08-07T15:09:47Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2517 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:09:47 dist-test-slave-3kk6 krb5kdc[21236](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:09:47 dist-test-slave-3kk6 krb5kdc[21236](info): set up 2 sockets
Aug 07 15:09:47 dist-test-slave-3kk6 krb5kdc[21236](info): commencing operation
krb5kdc: starting...
W20260807 15:09:49.035547 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:09:49 dist-test-slave-3kk6 krb5kdc[21236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:09:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:09:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:09:49.175828 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:42431
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:37759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:42431
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:49.294888 21254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:49.295195 21254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:49.295270 21254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:49.300189 21254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:49.300288 21254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:49.300326 21254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:49.300367 21254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:49.300395 21254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:09:49.306499 21254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:37759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:42431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:42431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:49.307783 21254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:49.308919 21254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:09:49.318486 21254 server_base.cc:1034] running on GCE node
W20260807 15:09:49.318425 21264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:49.318462 21260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:49.318425 21261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:49.319341 21254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:49.320574 21254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:49.321755 21254 hybrid_clock.cc:648] HybridClock initialized: now 1786115389321726 us; error 43 us; skew 500 ppm
Aug 07 15:09:49 dist-test-slave-3kk6 krb5kdc[21236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115389, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:49.328589 21254 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:09:49.329703 21254 webserver.cc:460] Webserver started at http://127.15.226.126:36373/ using document root <none> and password file <none>
I20260807 15:09:49.330331 21254 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:49.330410 21254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:49.330734 21254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:49.333027 21254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "d414e37a6fe446c985db56566fc4a4ca"
format_stamp: "Formatted at 2026-08-07 15:09:49 on dist-test-slave-3kk6"
server_key: "48944823d49f0c60a3ac9bb9ed6b63e5"
server_key_iv: "bdae9f5aa9d8be56cfa3bb93a5af83e9"
server_key_version: "encryptionkey@0"
I20260807 15:09:49.333688 21254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "d414e37a6fe446c985db56566fc4a4ca"
format_stamp: "Formatted at 2026-08-07 15:09:49 on dist-test-slave-3kk6"
server_key: "48944823d49f0c60a3ac9bb9ed6b63e5"
server_key_iv: "bdae9f5aa9d8be56cfa3bb93a5af83e9"
server_key_version: "encryptionkey@0"
I20260807 15:09:49.337957 21254 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:09:49.340756 21273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:49.342123 21254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:09:49.342305 21254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "d414e37a6fe446c985db56566fc4a4ca"
format_stamp: "Formatted at 2026-08-07 15:09:49 on dist-test-slave-3kk6"
server_key: "48944823d49f0c60a3ac9bb9ed6b63e5"
server_key_iv: "bdae9f5aa9d8be56cfa3bb93a5af83e9"
server_key_version: "encryptionkey@0"
I20260807 15:09:49.342489 21254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:49.361510 21254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:49.365363 21254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:49.365608 21254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:49.373737 21254 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:42431
I20260807 15:09:49.373757 21343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:42431 every 8 connection(s)
I20260807 15:09:49.374924 21254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:09:49.377820 21344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:49.383811 21344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d414e37a6fe446c985db56566fc4a4ca: Bootstrap starting.
I20260807 15:09:49.384344 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 21254
I20260807 15:09:49.384481 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:09:49.384759 16265 external_mini_cluster.cc:1428] Setting key 62be6209feb5264a8986b193c74149cf
I20260807 15:09:49.386976 21344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d414e37a6fe446c985db56566fc4a4ca: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:49.388052 21344 log.cc:824] T 00000000000000000000000000000000 P d414e37a6fe446c985db56566fc4a4ca: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:49.390833 21344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d414e37a6fe446c985db56566fc4a4ca: No bootstrap required, opened a new log
Aug 07 15:09:49 dist-test-slave-3kk6 krb5kdc[21236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:49.395889 21344 raft_consensus.cc:348] T 00000000000000000000000000000000 P d414e37a6fe446c985db56566fc4a4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d414e37a6fe446c985db56566fc4a4ca" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 42431 } }
I20260807 15:09:49.396250 21344 raft_consensus.cc:374] T 00000000000000000000000000000000 P d414e37a6fe446c985db56566fc4a4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:49.396338 21344 raft_consensus.cc:729] T 00000000000000000000000000000000 P d414e37a6fe446c985db56566fc4a4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d414e37a6fe446c985db56566fc4a4ca, State: Initialized, Role: FOLLOWER
I20260807 15:09:49.396955 21344 consensus_queue.cc:260] T 00000000000000000000000000000000 P d414e37a6fe446c985db56566fc4a4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d414e37a6fe446c985db56566fc4a4ca" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 42431 } }
I20260807 15:09:49.397164 21344 raft_consensus.cc:388] T 00000000000000000000000000000000 P d414e37a6fe446c985db56566fc4a4ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:09:49.397274 21344 raft_consensus.cc:482] T 00000000000000000000000000000000 P d414e37a6fe446c985db56566fc4a4ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:09:49.397408 21344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d414e37a6fe446c985db56566fc4a4ca [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:49.398519 21344 raft_consensus.cc:504] T 00000000000000000000000000000000 P d414e37a6fe446c985db56566fc4a4ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d414e37a6fe446c985db56566fc4a4ca" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 42431 } }
I20260807 15:09:49.398954 21344 leader_election.cc:302] T 00000000000000000000000000000000 P d414e37a6fe446c985db56566fc4a4ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d414e37a6fe446c985db56566fc4a4ca; no voters:
I20260807 15:09:49.399398 21344 leader_election.cc:288] T 00000000000000000000000000000000 P d414e37a6fe446c985db56566fc4a4ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:09:49.399534 21355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d414e37a6fe446c985db56566fc4a4ca [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:49.399777 21355 raft_consensus.cc:686] T 00000000000000000000000000000000 P d414e37a6fe446c985db56566fc4a4ca [term 1 LEADER]: Becoming Leader. State: Replica: d414e37a6fe446c985db56566fc4a4ca, State: Running, Role: LEADER
I20260807 15:09:49.400116 21355 consensus_queue.cc:237] T 00000000000000000000000000000000 P d414e37a6fe446c985db56566fc4a4ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d414e37a6fe446c985db56566fc4a4ca" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 42431 } }
I20260807 15:09:49.400946 21344 sys_catalog.cc:564] T 00000000000000000000000000000000 P d414e37a6fe446c985db56566fc4a4ca [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:49.402586 21356 sys_catalog.cc:455] T 00000000000000000000000000000000 P d414e37a6fe446c985db56566fc4a4ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d414e37a6fe446c985db56566fc4a4ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d414e37a6fe446c985db56566fc4a4ca" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 42431 } } }
I20260807 15:09:49.402745 21356 sys_catalog.cc:458] T 00000000000000000000000000000000 P d414e37a6fe446c985db56566fc4a4ca [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:49.403070 21357 sys_catalog.cc:455] T 00000000000000000000000000000000 P d414e37a6fe446c985db56566fc4a4ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader d414e37a6fe446c985db56566fc4a4ca. Latest consensus state: current_term: 1 leader_uuid: "d414e37a6fe446c985db56566fc4a4ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d414e37a6fe446c985db56566fc4a4ca" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 42431 } } }
I20260807 15:09:49.403162 21357 sys_catalog.cc:458] T 00000000000000000000000000000000 P d414e37a6fe446c985db56566fc4a4ca [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:49.403240 21349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:49.386313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52458 (local address 127.15.226.126:42431)
0807 15:09:49.386991 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:49.387015 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:49.387065 (+ 50us) server_negotiation.cc:407] Connection header received
0807 15:09:49.388146 (+ 1081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:49.388175 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:49.388672 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:49.389016 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:49.389915 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:49.391590 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:49.392541 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:49.393389 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:49.393595 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:49.396876 (+ 3281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:49.396883 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:49.396902 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:49.396942 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:49.399217 (+ 2275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:49.399871 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:49.399878 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:49.399885 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:49.400044 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:49.400405 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:49.400408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:49.400410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:49.401007 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:49.401214 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:49.401715 (+ 501us) server_negotiation.cc:299] Negotiation successful
0807 15:09:49.402063 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":143,"threads_started":1}
W20260807 15:09:49.406584 21376 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d414e37a6fe446c985db56566fc4a4ca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:09:49.406703 21376 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:09:49.406785 21369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:09:49.407866 21369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:09:49.414153 21369 catalog_manager.cc:1347] Generated new cluster ID: 47bfd6d315804ae68c82e5c13261e7b2
I20260807 15:09:49.414250 21369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:09:49.436580 21369 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:09:49.437670 21369 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:09:49.445292 21369 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d414e37a6fe446c985db56566fc4a4ca: Generated new TSK 0
I20260807 15:09:49.446115 21369 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.226.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
I20260807 15:09:49.486287 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:42431
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:37759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:49.741539 21383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:49.741847 21383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:49.741894 21383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:49.746117 21383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:49.746196 21383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:49.746263 21383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:09:49.752094 21383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:37759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:42431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:49.753338 21383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:49.754397 21383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:49.766537 21393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:49.766537 21390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:49.766558 21389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:49.766957 21383 server_base.cc:1034] running on GCE node
I20260807 15:09:49.767418 21383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:49.768149 21383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:49.769385 21383 hybrid_clock.cc:648] HybridClock initialized: now 1786115389769359 us; error 49 us; skew 500 ppm
Aug 07 15:09:49 dist-test-slave-3kk6 krb5kdc[21236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115389, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:49.772833 21383 init.cc:377] Logged in from keytab as kudu/127.15.226.65@KRBTEST.COM (short username kudu)
I20260807 15:09:49.773828 21383 webserver.cc:460] Webserver started at http://127.15.226.65:43625/ using document root <none> and password file <none>
I20260807 15:09:49.774451 21383 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:49.774514 21383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:49.774809 21383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:49.777110 21383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "a498eab836974a82a5f78a68ac0c9393"
format_stamp: "Formatted at 2026-08-07 15:09:49 on dist-test-slave-3kk6"
server_key: "250186c3e9bfd2443dc82c2807f4ba43"
server_key_iv: "acdb8a0eda7d2da70ec3a64bc891b73a"
server_key_version: "encryptionkey@0"
I20260807 15:09:49.777691 21383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "a498eab836974a82a5f78a68ac0c9393"
format_stamp: "Formatted at 2026-08-07 15:09:49 on dist-test-slave-3kk6"
server_key: "250186c3e9bfd2443dc82c2807f4ba43"
server_key_iv: "acdb8a0eda7d2da70ec3a64bc891b73a"
server_key_version: "encryptionkey@0"
I20260807 15:09:49.782059 21383 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 15:09:49.784873 21403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:49.786178 21383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:09:49.786340 21383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "a498eab836974a82a5f78a68ac0c9393"
format_stamp: "Formatted at 2026-08-07 15:09:49 on dist-test-slave-3kk6"
server_key: "250186c3e9bfd2443dc82c2807f4ba43"
server_key_iv: "acdb8a0eda7d2da70ec3a64bc891b73a"
server_key_version: "encryptionkey@0"
I20260807 15:09:49.786461 21383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:49.808353 21383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:49.811776 21383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:49.812042 21383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:49.812763 21383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:49.813757 21383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:49.813838 21383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:49.813916 21383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:49.813961 21383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:49.825242 21383 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:46473
I20260807 15:09:49.825284 21566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:46473 every 8 connection(s)
I20260807 15:09:49.826295 21383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:09:49.831764 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 21383
I20260807 15:09:49.831941 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:09:49.832337 16265 external_mini_cluster.cc:1428] Setting key 0f2bace9c395f86e17e206022dde9069
Aug 07 15:09:49 dist-test-slave-3kk6 krb5kdc[21236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115389, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:49.843292 21349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:49.828488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:60779 (local address 127.15.226.126:42431)
0807 15:09:49.828650 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:49.828654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:49.829581 (+ 927us) server_negotiation.cc:407] Connection header received
0807 15:09:49.831126 (+ 1545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:49.831129 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:49.831192 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:49.831280 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:49.833045 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:49.833634 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:49.835218 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:49.835672 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:49.835785 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:49.839052 (+ 3267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:49.839058 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:49.839060 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:49.839086 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:49.840739 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:49.841458 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:49.841460 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:49.841462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:49.841538 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:49.841984 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:49.841986 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:49.841987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:49.842189 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:49.842269 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:49.843028 (+ 759us) server_negotiation.cc:299] Negotiation successful
0807 15:09:49.843142 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:09:49.844627 21571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:49.828858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:42431 (local address 127.15.226.65:60779)
0807 15:09:49.829365 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:49.829414 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:49.830898 (+ 1484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:49.831497 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:49.831508 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:49.832118 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:49.832878 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:49.832895 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:49.833791 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:49.833795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:49.835066 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:49.835079 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:49.835886 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:49.835893 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:49.836085 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:49.836782 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:49.836809 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:49.838867 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:49.840873 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:49.840880 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:49.840897 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:49.841336 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:49.841638 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:49.841643 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:49.841648 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:49.841869 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:49.842305 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:49.842315 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:49.842732 (+ 417us) client_negotiation.cc:769] Sending connection context
0807 15:09:49.842976 (+ 244us) client_negotiation.cc:240] Negotiation successful
0807 15:09:49.843388 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":138,"threads_started":1}
WARNING: no policy specified for kudu/127.15.226.66@KRBTEST.COM; defaulting to no policy
I20260807 15:09:49.846226 21568 heartbeater.cc:347] Connected to a master server at 127.15.226.126:42431
I20260807 15:09:49.846503 21568 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:49.847236 21568 heartbeater.cc:511] Master 127.15.226.126:42431 requested a full tablet report, sending...
I20260807 15:09:49.849040 21301 ts_manager.cc:194] Registered new tserver with Master: a498eab836974a82a5f78a68ac0c9393 (127.15.226.65:46473)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.66@KRBTEST.COM" created.
I20260807 15:09:49.850889 21301 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.65@KRBTEST.COM'} at 127.15.226.65:60779
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
I20260807 15:09:49.866285 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:42431
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:37759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:49.995347 21574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:49.995653 21574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:49.995699 21574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:50.000022 21574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:50.000128 21574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:50.000193 21574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:09:50.006062 21574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:37759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:42431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:50.007279 21574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:50.008327 21574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:50.018775 21582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:50.018759 21581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:50.018851 21586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:50.019171 21574 server_base.cc:1034] running on GCE node
I20260807 15:09:50.019580 21574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:50.020377 21574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:50.021553 21574 hybrid_clock.cc:648] HybridClock initialized: now 1786115390021514 us; error 55 us; skew 500 ppm
Aug 07 15:09:50 dist-test-slave-3kk6 krb5kdc[21236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115390, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:50.024662 21574 init.cc:377] Logged in from keytab as kudu/127.15.226.66@KRBTEST.COM (short username kudu)
I20260807 15:09:50.025609 21574 webserver.cc:460] Webserver started at http://127.15.226.66:38183/ using document root <none> and password file <none>
I20260807 15:09:50.026150 21574 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:50.026211 21574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:50.026381 21574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:50.028537 21574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "7366bcfa91214350a4dd4dca9d28c2da"
format_stamp: "Formatted at 2026-08-07 15:09:50 on dist-test-slave-3kk6"
server_key: "94d7106509ad8d13f2fa55724702069d"
server_key_iv: "57b9ba6e59d2512604bfb46ce523e41f"
server_key_version: "encryptionkey@0"
I20260807 15:09:50.029093 21574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "7366bcfa91214350a4dd4dca9d28c2da"
format_stamp: "Formatted at 2026-08-07 15:09:50 on dist-test-slave-3kk6"
server_key: "94d7106509ad8d13f2fa55724702069d"
server_key_iv: "57b9ba6e59d2512604bfb46ce523e41f"
server_key_version: "encryptionkey@0"
I20260807 15:09:50.032953 21574 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:09:50.035421 21594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:50.036652 21574 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:09:50.036795 21574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "7366bcfa91214350a4dd4dca9d28c2da"
format_stamp: "Formatted at 2026-08-07 15:09:50 on dist-test-slave-3kk6"
server_key: "94d7106509ad8d13f2fa55724702069d"
server_key_iv: "57b9ba6e59d2512604bfb46ce523e41f"
server_key_version: "encryptionkey@0"
I20260807 15:09:50.036931 21574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:50.055094 21574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:50.058389 21574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:50.058619 21574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:50.059325 21574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:50.060216 21574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:50.060292 21574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:50.060369 21574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:50.060411 21574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:50.070636 21574 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:43815
I20260807 15:09:50.070675 21752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:43815 every 8 connection(s)
I20260807 15:09:50.071674 21574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:09:50.074754 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 21574
I20260807 15:09:50.074882 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:09:50.075168 16265 external_mini_cluster.cc:1428] Setting key befd3a4f2387a739d8d07f586d282cb7
Aug 07 15:09:50 dist-test-slave-3kk6 krb5kdc[21236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115390, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.226.67@KRBTEST.COM; defaulting to no policy
I20260807 15:09:50.087720 21349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:50.073593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:38341 (local address 127.15.226.126:42431)
0807 15:09:50.073745 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:50.073749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:50.074618 (+ 869us) server_negotiation.cc:407] Connection header received
0807 15:09:50.076348 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:50.076351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:50.076394 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:50.076504 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:50.077831 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:50.078711 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:50.080055 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:50.080301 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:50.080454 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:50.083296 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:50.083302 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:50.083308 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:50.083349 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:50.085254 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:50.085801 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:50.085807 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:50.085810 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:50.085908 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:50.086265 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:50.086271 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:50.086274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:50.086557 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:50.086705 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:50.087481 (+ 776us) server_negotiation.cc:299] Negotiation successful
0807 15:09:50.087590 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:09:50.088707 21755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:50.073963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:42431 (local address 127.15.226.66:38341)
0807 15:09:50.074460 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:50.074505 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:50.076113 (+ 1608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:50.076722 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:50.076733 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:50.077141 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:50.077671 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:50.077685 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:50.078918 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:50.078921 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:50.079898 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:50.079910 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:50.080536 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:50.080539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:50.080704 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:50.081423 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:50.081440 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:50.083087 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:50.085388 (+ 2301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:50.085394 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:50.085409 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:50.085689 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:50.086011 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:50.086013 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:50.086014 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:50.086157 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:50.086924 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:50.086930 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:50.087237 (+ 307us) client_negotiation.cc:769] Sending connection context
0807 15:09:50.087412 (+ 175us) client_negotiation.cc:240] Negotiation successful
0807 15:09:50.087703 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":123,"threads_started":1}
I20260807 15:09:50.090062 21753 heartbeater.cc:347] Connected to a master server at 127.15.226.126:42431
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.67@KRBTEST.COM" created.
I20260807 15:09:50.090333 21753 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:50.090976 21753 heartbeater.cc:511] Master 127.15.226.126:42431 requested a full tablet report, sending...
I20260807 15:09:50.092135 21301 ts_manager.cc:194] Registered new tserver with Master: 7366bcfa91214350a4dd4dca9d28c2da (127.15.226.66:43815)
I20260807 15:09:50.092737 21301 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.66@KRBTEST.COM'} at 127.15.226.66:38341
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
I20260807 15:09:50.110546 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:42431
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:37759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:50.234634 21758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:50.235013 21758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:50.235095 21758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:50.239319 21758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:50.239398 21758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:50.239493 21758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:09:50.245096 21758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:37759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:42431
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:50.246490 21758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:50.247752 21758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:50.258116 21769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:50.258180 21767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:50.258242 21766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:50.258610 21758 server_base.cc:1034] running on GCE node
I20260807 15:09:50.259070 21758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:50.259763 21758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:50.260936 21758 hybrid_clock.cc:648] HybridClock initialized: now 1786115390260920 us; error 28 us; skew 500 ppm
Aug 07 15:09:50 dist-test-slave-3kk6 krb5kdc[21236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115390, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:50.263901 21758 init.cc:377] Logged in from keytab as kudu/127.15.226.67@KRBTEST.COM (short username kudu)
I20260807 15:09:50.265009 21758 webserver.cc:460] Webserver started at http://127.15.226.67:35205/ using document root <none> and password file <none>
I20260807 15:09:50.265606 21758 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:50.265672 21758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:50.265931 21758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:50.268158 21758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "a4159f8dc6db49d0a10d54eb25307203"
format_stamp: "Formatted at 2026-08-07 15:09:50 on dist-test-slave-3kk6"
server_key: "15ef579a5746720d4295e6b9a627ff50"
server_key_iv: "b851e1ad1625e7d3f7d1bb4303d7ae89"
server_key_version: "encryptionkey@0"
I20260807 15:09:50.268767 21758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "a4159f8dc6db49d0a10d54eb25307203"
format_stamp: "Formatted at 2026-08-07 15:09:50 on dist-test-slave-3kk6"
server_key: "15ef579a5746720d4295e6b9a627ff50"
server_key_iv: "b851e1ad1625e7d3f7d1bb4303d7ae89"
server_key_version: "encryptionkey@0"
I20260807 15:09:50.272756 21758 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:09:50.275295 21779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:50.276562 21758 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:09:50.276688 21758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "a4159f8dc6db49d0a10d54eb25307203"
format_stamp: "Formatted at 2026-08-07 15:09:50 on dist-test-slave-3kk6"
server_key: "15ef579a5746720d4295e6b9a627ff50"
server_key_iv: "b851e1ad1625e7d3f7d1bb4303d7ae89"
server_key_version: "encryptionkey@0"
I20260807 15:09:50.276805 21758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:50.299978 21758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:50.304304 21758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:50.304575 21758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:50.305299 21758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:50.306133 21758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:50.306190 21758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:50.306231 21758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:50.306252 21758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:50.316247 21758 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:38035
I20260807 15:09:50.316279 21936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:38035 every 8 connection(s)
I20260807 15:09:50.317247 21758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:09:50.319325 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 21758
I20260807 15:09:50.319468 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:09:50.319783 16265 external_mini_cluster.cc:1428] Setting key 3fc57db07d6c582768bfcc938c0dd57a
Aug 07 15:09:50 dist-test-slave-3kk6 krb5kdc[21236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115390, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:50.332191 21349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:50.319435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:42869 (local address 127.15.226.126:42431)
0807 15:09:50.319593 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:50.319597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:50.320696 (+ 1099us) server_negotiation.cc:407] Connection header received
0807 15:09:50.322539 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:50.322542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:50.322601 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:50.322719 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:50.323951 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:50.324476 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:50.325599 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:50.325807 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:50.325885 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:50.328653 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:50.328656 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:50.328659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:50.328684 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:50.330209 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:50.330739 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:50.330741 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:50.330742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:50.330825 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:50.331149 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:50.331151 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:50.331153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:50.331347 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:50.331432 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:50.331954 (+ 522us) server_negotiation.cc:299] Negotiation successful
0807 15:09:50.332061 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 15:09:50.333223 21942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:50.319932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:42431 (local address 127.15.226.67:42869)
0807 15:09:50.320498 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:50.320581 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:50.322306 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:50.322906 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:50.322913 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:50.323291 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:50.323809 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:50.323824 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:50.324589 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:50.324592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:50.325492 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:50.325501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:50.325940 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:50.325942 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:50.326094 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:50.326774 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:50.326799 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:50.328463 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:50.330317 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:50.330322 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:50.330335 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:50.330597 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:50.330910 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:50.330912 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:50.330913 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:50.331060 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:50.331441 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:50.331447 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:50.331713 (+ 266us) client_negotiation.cc:769] Sending connection context
0807 15:09:50.331907 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 15:09:50.332186 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":153,"threads_started":1}
I20260807 15:09:50.334619 21937 heartbeater.cc:347] Connected to a master server at 127.15.226.126:42431
I20260807 15:09:50.334940 21937 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:50.335503 21937 heartbeater.cc:511] Master 127.15.226.126:42431 requested a full tablet report, sending...
I20260807 15:09:50.336627 21300 ts_manager.cc:194] Registered new tserver with Master: a4159f8dc6db49d0a10d54eb25307203 (127.15.226.67:38035)
I20260807 15:09:50.337222 21300 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.67@KRBTEST.COM'} at 127.15.226.67:42869
I20260807 15:09:50.346714 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 15:09:50 dist-test-slave-3kk6 krb5kdc[21236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 15:09:50 dist-test-slave-3kk6 krb5kdc[21236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.65@KRBTEST.COM
I20260807 15:09:50.376718 21950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:50.366020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36276 (local address 127.15.226.65:46473)
0807 15:09:50.366406 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:50.366412 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:50.366442 (+ 30us) server_negotiation.cc:407] Connection header received
0807 15:09:50.366497 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:50.366502 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:50.366764 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:50.366941 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:50.367724 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:50.368522 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:50.369431 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:50.369827 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:50.369957 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:50.372661 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:50.372672 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:50.372686 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:50.372713 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:50.374813 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:50.375232 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:50.375238 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:50.375240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:50.375348 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:50.375611 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:50.375614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:50.375616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:50.375946 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:50.376153 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:50.376366 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 15:09:50.376535 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":156,"threads_started":1}
Aug 07 15:09:50 dist-test-slave-3kk6 krb5kdc[21236](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:50.389590 21349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:50.380011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52460 (local address 127.15.226.126:42431)
0807 15:09:50.380141 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:50.380145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:50.380209 (+ 64us) server_negotiation.cc:407] Connection header received
0807 15:09:50.380328 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:50.380331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:50.380378 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:50.380463 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:50.381131 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:50.381989 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:50.383029 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:50.383356 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:50.383469 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:50.386139 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:50.386142 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:50.386145 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:50.386175 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:50.387837 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:50.388261 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:50.388265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:50.388267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:50.388358 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:50.388633 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:50.388636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:50.388638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:50.388934 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:50.389046 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:50.389341 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 15:09:50.389453 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 15:09:50.390054 21300 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52460
I20260807 15:09:50.390763 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 21383
I20260807 15:09:50.412884 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 21574
I20260807 15:09:50.426430 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 21758
I20260807 15:09:50.436703 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 21254
2026-08-07T15:09:50Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2655 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:09:50 dist-test-slave-3kk6 krb5kdc[21958](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:09:50 dist-test-slave-3kk6 krb5kdc[21958](info): set up 2 sockets
Aug 07 15:09:50 dist-test-slave-3kk6 krb5kdc[21958](info): commencing operation
krb5kdc: starting...
W20260807 15:09:51.680723 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:09:51 dist-test-slave-3kk6 krb5kdc[21958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:09:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:09:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:09:51.822393 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:45741
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:36639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:45741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:51.941856 21976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:51.942260 21976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:51.942364 21976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:51.947345 21976 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:51.947453 21976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:51.947491 21976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:51.947551 21976 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:51.947608 21976 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:09:51.953429 21976 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:36639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:45741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:45741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:51.954787 21976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:51.955897 21976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:51.964905 21984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:51.964903 21981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:51.964923 21982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:51.965564 21976 server_base.cc:1034] running on GCE node
I20260807 15:09:51.966110 21976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:51.967397 21976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:51.968569 21976 hybrid_clock.cc:648] HybridClock initialized: now 1786115391968539 us; error 45 us; skew 500 ppm
Aug 07 15:09:51 dist-test-slave-3kk6 krb5kdc[21958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115391, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:51.971904 21976 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:09:51.972646 21976 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:51.972721 21976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:51.972896 21976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:51.975150 21976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "f5be38c90c754b24abe2f3fb4b2be231"
format_stamp: "Formatted at 2026-08-07 15:09:51 on dist-test-slave-3kk6"
server_key: "157e5e77c3aba8eb8d9a838c57825279"
server_key_iv: "17c385d5df1a9cde268602096aa1dc35"
server_key_version: "encryptionkey@0"
I20260807 15:09:51.975730 21976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "f5be38c90c754b24abe2f3fb4b2be231"
format_stamp: "Formatted at 2026-08-07 15:09:51 on dist-test-slave-3kk6"
server_key: "157e5e77c3aba8eb8d9a838c57825279"
server_key_iv: "17c385d5df1a9cde268602096aa1dc35"
server_key_version: "encryptionkey@0"
I20260807 15:09:51.979713 21976 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:09:51.982424 21991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:51.983889 21976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 15:09:51.984045 21976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "f5be38c90c754b24abe2f3fb4b2be231"
format_stamp: "Formatted at 2026-08-07 15:09:51 on dist-test-slave-3kk6"
server_key: "157e5e77c3aba8eb8d9a838c57825279"
server_key_iv: "17c385d5df1a9cde268602096aa1dc35"
server_key_version: "encryptionkey@0"
I20260807 15:09:51.984172 21976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:52.021031 21976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:52.024904 21976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:52.025156 21976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:52.032965 21976 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:45741
I20260807 15:09:52.033272 22066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:45741 every 8 connection(s)
I20260807 15:09:52.033820 21976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:09:52.036824 22068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:52.041302 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 21976
I20260807 15:09:52.041486 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:09:52.041774 16265 external_mini_cluster.cc:1428] Setting key 3f54745de98182c1a7b0a9a67da87853
I20260807 15:09:52.043474 22068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231: Bootstrap starting.
I20260807 15:09:52.046545 22068 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:52.047595 22068 log.cc:824] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:52.050015 22068 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231: No bootstrap required, opened a new log
Aug 07 15:09:52 dist-test-slave-3kk6 krb5kdc[21958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:52.054746 22068 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5be38c90c754b24abe2f3fb4b2be231" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45741 } }
I20260807 15:09:52.055132 22068 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:52.055220 22068 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5be38c90c754b24abe2f3fb4b2be231, State: Initialized, Role: FOLLOWER
I20260807 15:09:52.055881 22068 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5be38c90c754b24abe2f3fb4b2be231" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45741 } }
I20260807 15:09:52.056102 22068 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:09:52.056208 22068 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:09:52.056342 22068 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:52.057408 22068 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5be38c90c754b24abe2f3fb4b2be231" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45741 } }
I20260807 15:09:52.057843 22068 leader_election.cc:302] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5be38c90c754b24abe2f3fb4b2be231; no voters:
I20260807 15:09:52.058296 22068 leader_election.cc:288] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:09:52.058383 22073 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:52.058598 22073 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231 [term 1 LEADER]: Becoming Leader. State: Replica: f5be38c90c754b24abe2f3fb4b2be231, State: Running, Role: LEADER
I20260807 15:09:52.059074 22071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:52.043339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52414 (local address 127.15.226.126:45741)
0807 15:09:52.043840 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:52.043861 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:52.043904 (+ 43us) server_negotiation.cc:407] Connection header received
0807 15:09:52.044807 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:52.044836 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:52.045297 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:52.045668 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:52.046475 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.048050 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.049144 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.049832 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.050004 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:52.052773 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:52.052784 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:52.052809 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:52.052855 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:52.055073 (+ 2218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:52.055690 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:52.055696 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:52.055703 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:52.055856 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:52.056271 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:52.056275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:52.056276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:52.056882 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:52.057122 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:52.057451 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 15:09:52.057997 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":177,"threads_started":1}
I20260807 15:09:52.059089 22073 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5be38c90c754b24abe2f3fb4b2be231" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45741 } }
I20260807 15:09:52.059800 22068 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:52.061245 22076 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5be38c90c754b24abe2f3fb4b2be231. Latest consensus state: current_term: 1 leader_uuid: "f5be38c90c754b24abe2f3fb4b2be231" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5be38c90c754b24abe2f3fb4b2be231" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45741 } } }
I20260807 15:09:52.061385 22076 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:52.061828 22075 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5be38c90c754b24abe2f3fb4b2be231" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5be38c90c754b24abe2f3fb4b2be231" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45741 } } }
I20260807 15:09:52.061944 22075 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231 [sys.catalog]: This master's current role is: LEADER
W20260807 15:09:52.065773 22094 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:09:52.065853 22094 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:09:52.065939 22085 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:09:52.066879 22085 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:09:52.073629 22085 catalog_manager.cc:1347] Generated new cluster ID: c873c72c996643a3b273d5afe85380fc
I20260807 15:09:52.073721 22085 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:09:52.092733 22085 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:09:52.093751 22085 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:09:52.102075 22085 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231: Generated new TSK 0
I20260807 15:09:52.103111 22085 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.226.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
I20260807 15:09:52.145524 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:45741
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:36639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:52.268116 22100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:52.268426 22100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:52.268509 22100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:52.272877 22100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:52.272961 22100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:52.273028 22100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:09:52.278664 22100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:36639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:45741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:52.279870 22100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:52.280984 22100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:52.291659 22107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:52.291746 22110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:52.291661 22108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:52.292009 22100 server_base.cc:1034] running on GCE node
I20260807 15:09:52.292563 22100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:52.293273 22100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:52.294458 22100 hybrid_clock.cc:648] HybridClock initialized: now 1786115392294423 us; error 53 us; skew 500 ppm
Aug 07 15:09:52 dist-test-slave-3kk6 krb5kdc[21958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115392, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:52.297740 22100 init.cc:377] Logged in from keytab as kudu/127.15.226.65@KRBTEST.COM (short username kudu)
I20260807 15:09:52.298596 22100 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:52.298772 22100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:52.299034 22100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:52.301290 22100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "1a8482df93db4aff9971b525cbae193e"
format_stamp: "Formatted at 2026-08-07 15:09:52 on dist-test-slave-3kk6"
server_key: "4fc20deddc92018249a4c672bbd1a29f"
server_key_iv: "99c223228721c5056c0e081b7ae446e1"
server_key_version: "encryptionkey@0"
I20260807 15:09:52.301947 22100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "1a8482df93db4aff9971b525cbae193e"
format_stamp: "Formatted at 2026-08-07 15:09:52 on dist-test-slave-3kk6"
server_key: "4fc20deddc92018249a4c672bbd1a29f"
server_key_iv: "99c223228721c5056c0e081b7ae446e1"
server_key_version: "encryptionkey@0"
I20260807 15:09:52.306339 22100 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:09:52.309320 22120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:52.311429 22100 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:09:52.311618 22100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "1a8482df93db4aff9971b525cbae193e"
format_stamp: "Formatted at 2026-08-07 15:09:52 on dist-test-slave-3kk6"
server_key: "4fc20deddc92018249a4c672bbd1a29f"
server_key_iv: "99c223228721c5056c0e081b7ae446e1"
server_key_version: "encryptionkey@0"
I20260807 15:09:52.311807 22100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:52.331002 22100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:52.334578 22100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:52.334852 22100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:52.335469 22100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:52.336328 22100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:52.336380 22100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:52.336422 22100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:52.336438 22100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:52.346761 22100 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:44213
I20260807 15:09:52.346850 22278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:44213 every 8 connection(s)
I20260807 15:09:52.347503 22100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:09:52.354526 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 22100
I20260807 15:09:52.354712 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:09:52.355026 16265 external_mini_cluster.cc:1428] Setting key 65e827c7f6b82ba8638eec5891fb88b5
Aug 07 15:09:52 dist-test-slave-3kk6 krb5kdc[21958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115392, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:52.364282 22071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:52.349574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:59033 (local address 127.15.226.126:45741)
0807 15:09:52.349772 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:52.349777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:52.351241 (+ 1464us) server_negotiation.cc:407] Connection header received
0807 15:09:52.352195 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:52.352199 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:52.352251 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:52.352342 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:52.353652 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.354352 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.355989 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.356234 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.356358 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:52.359307 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:52.359313 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:52.359318 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:52.359356 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:52.361242 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:52.362091 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:52.362098 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:52.362102 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:52.362209 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:52.362780 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:52.362784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:52.362786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:52.363085 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:52.363207 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:52.363855 (+ 648us) server_negotiation.cc:299] Negotiation successful
0807 15:09:52.364019 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 15:09:52.365267 22281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:52.349988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:45741 (local address 127.15.226.65:59033)
0807 15:09:52.351071 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:52.351119 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:52.351963 (+ 844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:52.352521 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:52.352528 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:52.352929 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:52.353465 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:52.353480 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.354538 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.354541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:52.355835 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:52.355847 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.356414 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.356418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:52.356600 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:52.357317 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:52.357336 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:52.359056 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:52.361514 (+ 2458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:52.361524 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:52.361544 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:52.361967 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:52.362370 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:52.362377 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:52.362381 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:52.362591 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:52.363232 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:52.363238 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:52.363554 (+ 316us) client_negotiation.cc:769] Sending connection context
0807 15:09:52.363741 (+ 187us) client_negotiation.cc:240] Negotiation successful
0807 15:09:52.364086 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":176,"threads_started":1}
WARNING: no policy specified for kudu/127.15.226.66@KRBTEST.COM; defaulting to no policy
I20260807 15:09:52.366899 22279 heartbeater.cc:347] Connected to a master server at 127.15.226.126:45741
I20260807 15:09:52.367203 22279 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:52.367930 22279 heartbeater.cc:511] Master 127.15.226.126:45741 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.66@KRBTEST.COM" created.
I20260807 15:09:52.370034 22014 ts_manager.cc:194] Registered new tserver with Master: 1a8482df93db4aff9971b525cbae193e (127.15.226.65:44213)
I20260807 15:09:52.371775 22014 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.65@KRBTEST.COM'} at 127.15.226.65:59033
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
I20260807 15:09:52.390334 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:45741
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:36639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:52.518869 22284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:52.519273 22284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:52.519374 22284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:52.524245 22284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:52.524362 22284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:52.524477 22284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:09:52.531065 22284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:36639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:45741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:52.532418 22284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:52.533631 22284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:52.544852 22292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:52.545032 22284 server_base.cc:1034] running on GCE node
W20260807 15:09:52.544808 22297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:52.544768 22294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:52.545722 22284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:52.546522 22284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:52.547791 22284 hybrid_clock.cc:648] HybridClock initialized: now 1786115392547718 us; error 46 us; skew 500 ppm
Aug 07 15:09:52 dist-test-slave-3kk6 krb5kdc[21958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115392, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:52.551385 22284 init.cc:377] Logged in from keytab as kudu/127.15.226.66@KRBTEST.COM (short username kudu)
I20260807 15:09:52.552367 22284 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:52.552500 22284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:52.552784 22284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:52.555378 22284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "0220f6ef067f4d26a0dd7cdd76dbbef9"
format_stamp: "Formatted at 2026-08-07 15:09:52 on dist-test-slave-3kk6"
server_key: "aa07d82946ae337f71e2542f9cd13fc7"
server_key_iv: "03505989321c6f4e1ea12f66aad1f876"
server_key_version: "encryptionkey@0"
I20260807 15:09:52.556110 22284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "0220f6ef067f4d26a0dd7cdd76dbbef9"
format_stamp: "Formatted at 2026-08-07 15:09:52 on dist-test-slave-3kk6"
server_key: "aa07d82946ae337f71e2542f9cd13fc7"
server_key_iv: "03505989321c6f4e1ea12f66aad1f876"
server_key_version: "encryptionkey@0"
I20260807 15:09:52.560714 22284 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:09:52.564023 22304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:52.565412 22284 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:09:52.565577 22284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "0220f6ef067f4d26a0dd7cdd76dbbef9"
format_stamp: "Formatted at 2026-08-07 15:09:52 on dist-test-slave-3kk6"
server_key: "aa07d82946ae337f71e2542f9cd13fc7"
server_key_iv: "03505989321c6f4e1ea12f66aad1f876"
server_key_version: "encryptionkey@0"
I20260807 15:09:52.565723 22284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:52.588578 22284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:52.592602 22284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:52.592862 22284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:52.593544 22284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:52.594485 22284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:52.594537 22284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:52.594578 22284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:52.594635 22284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:52.604818 22284 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:34089
I20260807 15:09:52.604836 22462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:34089 every 8 connection(s)
I20260807 15:09:52.605518 22284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:09:52.609545 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 22284
I20260807 15:09:52.609704 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:09:52.610093 16265 external_mini_cluster.cc:1428] Setting key 802df2036c8419555bc87e05b6fb15ed
Aug 07 15:09:52 dist-test-slave-3kk6 krb5kdc[21958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115392, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.226.67@KRBTEST.COM; defaulting to no policy
I20260807 15:09:52.622380 22071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:52.607456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:56619 (local address 127.15.226.126:45741)
0807 15:09:52.607609 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:52.607613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:52.608812 (+ 1199us) server_negotiation.cc:407] Connection header received
0807 15:09:52.609902 (+ 1090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:52.609907 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:52.609980 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:52.610083 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:52.611639 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.612236 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.614074 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.614495 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.614622 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:52.618011 (+ 3389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:52.618014 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:52.618016 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:52.618043 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:52.619647 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:52.620403 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:52.620410 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:52.620411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:52.620516 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:52.621004 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:52.621007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:52.621008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:52.621224 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:52.621317 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:52.622117 (+ 800us) server_negotiation.cc:299] Negotiation successful
0807 15:09:52.622247 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 15:09:52.623462 22465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:52.607860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:45741 (local address 127.15.226.66:56619)
0807 15:09:52.608579 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:52.608626 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:52.609581 (+ 955us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:52.610303 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:52.610310 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:52.610799 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:52.611490 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:52.611508 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.612386 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.612392 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:52.613913 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:52.613929 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.614767 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.614773 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:52.614926 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:52.615692 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:52.615710 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:52.617742 (+ 2032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:52.619783 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:52.619793 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:52.619818 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:52.620241 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:52.620639 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:52.620645 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:52.620650 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:52.620867 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:52.621361 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:52.621371 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:52.621747 (+ 376us) client_negotiation.cc:769] Sending connection context
0807 15:09:52.622053 (+ 306us) client_negotiation.cc:240] Negotiation successful
0807 15:09:52.622448 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":115,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.67@KRBTEST.COM" created.
I20260807 15:09:52.624864 22463 heartbeater.cc:347] Connected to a master server at 127.15.226.126:45741
I20260807 15:09:52.625150 22463 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:52.625795 22463 heartbeater.cc:511] Master 127.15.226.126:45741 requested a full tablet report, sending...
I20260807 15:09:52.627104 22014 ts_manager.cc:194] Registered new tserver with Master: 0220f6ef067f4d26a0dd7cdd76dbbef9 (127.15.226.66:34089)
I20260807 15:09:52.628078 22014 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.66@KRBTEST.COM'} at 127.15.226.66:56619
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
I20260807 15:09:52.642434 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:45741
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:36639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:52.766520 22468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:52.766861 22468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:52.766912 22468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:52.771451 22468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:52.771528 22468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:52.771591 22468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:09:52.777068 22468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:36639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:45741
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:52.778182 22468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:52.779242 22468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:52.789479 22484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:52.789505 22479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:52.789510 22481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:52.789665 22468 server_base.cc:1034] running on GCE node
I20260807 15:09:52.790294 22468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:52.791015 22468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:52.792204 22468 hybrid_clock.cc:648] HybridClock initialized: now 1786115392792192 us; error 32 us; skew 500 ppm
Aug 07 15:09:52 dist-test-slave-3kk6 krb5kdc[21958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115392, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:52.795384 22468 init.cc:377] Logged in from keytab as kudu/127.15.226.67@KRBTEST.COM (short username kudu)
I20260807 15:09:52.796209 22468 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:52.796279 22468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:52.796460 22468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:52.798611 22468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "072a42a89b4845f0956a593c0b889ef7"
format_stamp: "Formatted at 2026-08-07 15:09:52 on dist-test-slave-3kk6"
server_key: "953a270a4fb35362dd28e212158b5e3a"
server_key_iv: "3f730a6f51def276f55fe9c4e3f8a34f"
server_key_version: "encryptionkey@0"
I20260807 15:09:52.799273 22468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "072a42a89b4845f0956a593c0b889ef7"
format_stamp: "Formatted at 2026-08-07 15:09:52 on dist-test-slave-3kk6"
server_key: "953a270a4fb35362dd28e212158b5e3a"
server_key_iv: "3f730a6f51def276f55fe9c4e3f8a34f"
server_key_version: "encryptionkey@0"
I20260807 15:09:52.803369 22468 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:09:52.806125 22492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:52.807552 22468 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:09:52.807668 22468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "072a42a89b4845f0956a593c0b889ef7"
format_stamp: "Formatted at 2026-08-07 15:09:52 on dist-test-slave-3kk6"
server_key: "953a270a4fb35362dd28e212158b5e3a"
server_key_iv: "3f730a6f51def276f55fe9c4e3f8a34f"
server_key_version: "encryptionkey@0"
I20260807 15:09:52.807767 22468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:52.850149 22468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:52.853940 22468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:52.854125 22468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:52.854794 22468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:52.855679 22468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:52.855739 22468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:52.855783 22468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:52.855810 22468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:52.865975 22468 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:43853
I20260807 15:09:52.865995 22664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:43853 every 8 connection(s)
I20260807 15:09:52.866725 22468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:09:52.871950 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 22468
I20260807 15:09:52.872125 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:09:52.872531 16265 external_mini_cluster.cc:1428] Setting key bf100d2065997948f702c8383fa17410
Aug 07 15:09:52 dist-test-slave-3kk6 krb5kdc[21958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115392, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:52.883098 22071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:52.868628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:41067 (local address 127.15.226.126:45741)
0807 15:09:52.868789 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:52.868793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:52.870278 (+ 1485us) server_negotiation.cc:407] Connection header received
0807 15:09:52.871578 (+ 1300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:52.871582 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:52.871642 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:52.871743 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:52.873558 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.874351 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.875836 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.876073 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.876169 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:52.879116 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:52.879120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:52.879122 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:52.879150 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:52.880729 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:52.881318 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:52.881321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:52.881322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:52.881407 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:52.881755 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:52.881757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:52.881758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:52.882005 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:52.882085 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:52.882825 (+ 740us) server_negotiation.cc:299] Negotiation successful
0807 15:09:52.882955 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 15:09:52.884019 22669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:52.868985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:45741 (local address 127.15.226.67:41067)
0807 15:09:52.870028 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:52.870079 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:52.871295 (+ 1216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:52.872019 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:52.872033 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:52.872663 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:52.873343 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:52.873360 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.874500 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.874504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:52.875648 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:52.875660 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.876224 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.876227 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:52.876392 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:52.877111 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:52.877129 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:52.878888 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:52.880878 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:52.880884 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:52.880900 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:52.881206 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:52.881498 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:52.881500 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:52.881501 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:52.881657 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:52.882107 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:52.882113 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:52.882400 (+ 287us) client_negotiation.cc:769] Sending connection context
0807 15:09:52.882582 (+ 182us) client_negotiation.cc:240] Negotiation successful
0807 15:09:52.882983 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":188,"threads_started":1}
I20260807 15:09:52.885473 22667 heartbeater.cc:347] Connected to a master server at 127.15.226.126:45741
I20260807 15:09:52.885794 22667 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:52.886334 22667 heartbeater.cc:511] Master 127.15.226.126:45741 requested a full tablet report, sending...
I20260807 15:09:52.887800 22014 ts_manager.cc:194] Registered new tserver with Master: 072a42a89b4845f0956a593c0b889ef7 (127.15.226.67:43853)
I20260807 15:09:52.888384 22014 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.67@KRBTEST.COM'} at 127.15.226.67:41067
I20260807 15:09:52.899703 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:09:52.910719 22071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:52.902082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52428 (local address 127.15.226.126:45741)
0807 15:09:52.902287 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:52.902291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:52.902454 (+ 163us) server_negotiation.cc:407] Connection header received
0807 15:09:52.902632 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:52.902635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:52.902790 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:52.902886 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:52.903706 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.904264 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.905239 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.905468 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.905584 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:52.907194 (+ 1610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:52.907197 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:52.907200 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:52.907227 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:52.908758 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:52.909285 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:52.909294 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:52.909296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:52.909381 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:52.909682 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:52.909685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:52.909687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:52.909952 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:52.910102 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:52.910355 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 15:09:52.910505 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 15:09:52.914245 22014 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52428:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:09:52.917693 22014 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:09:52.933894 22687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:52.927183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:34089 (local address 127.0.0.1:46910)
0807 15:09:52.928223 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:52.928240 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:52.928335 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:52.929109 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:52.929115 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:52.929138 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:52.929283 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:52.929291 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.931140 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.931146 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:52.932210 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:52.932221 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.933419 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.933422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:52.933512 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:52.933548 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 15:09:52.933649 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 15:09:52.933757 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":965,"mutex_wait_us":2,"thread_start_us":124,"threads_started":1}
I20260807 15:09:52.934002 22686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:52.927583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46910 (local address 127.15.226.66:34089)
0807 15:09:52.928603 (+ 1020us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:52.928615 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:52.928638 (+ 23us) server_negotiation.cc:407] Connection header received
0807 15:09:52.928715 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:52.928719 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:52.928926 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:52.929112 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:52.929436 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.930770 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.932355 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.933261 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.933388 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:52.933560 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:52.933709 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 15:09:52.933831 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":906,"thread_start_us":119,"threads_started":1}
I20260807 15:09:52.934552 22684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:52.927906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:43853 (local address 127.0.0.1:60462)
0807 15:09:52.928282 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:52.928298 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:52.928392 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:52.929297 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:52.929300 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:52.929317 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:52.929429 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:52.929435 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.930479 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.930486 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:52.932301 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:52.932310 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.934134 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.934139 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:52.934234 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:52.934257 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 15:09:52.934351 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 15:09:52.934431 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":309,"thread_start_us":107,"threads_started":1}
I20260807 15:09:52.935139 22689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:52.928082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60462 (local address 127.15.226.67:43853)
0807 15:09:52.928603 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:52.928616 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:52.928638 (+ 22us) server_negotiation.cc:407] Connection header received
0807 15:09:52.928715 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:52.928720 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:52.928926 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:52.929112 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:52.929569 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.930329 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.932422 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.933273 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.933388 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:52.933560 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:52.934872 (+ 1312us) server_negotiation.cc:299] Negotiation successful
0807 15:09:52.934992 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":157,"threads_started":1}
I20260807 15:09:52.935129 22683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:52.927077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:44213 (local address 127.0.0.1:35190)
0807 15:09:52.927762 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:52.927820 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:52.927961 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:52.929307 (+ 1346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:52.929312 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:52.929335 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:52.929433 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:52.929438 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.930900 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.930904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:52.932298 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:52.932310 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.934737 (+ 2427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.934741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:52.934841 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:52.934866 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 15:09:52.934930 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 15:09:52.934994 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":52,"thread_start_us":123,"threads_started":1}
I20260807 15:09:52.935782 22685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:52.927228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35190 (local address 127.15.226.65:44213)
0807 15:09:52.927927 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:52.927934 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:52.927960 (+ 26us) server_negotiation.cc:407] Connection header received
0807 15:09:52.928710 (+ 750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:52.928716 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:52.928945 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:52.929112 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:52.929585 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.930759 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.933635 (+ 2876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:52.934497 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:52.934616 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:52.935491 (+ 875us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:52.935536 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 15:09:52.935643 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":128,"threads_started":1}
I20260807 15:09:52.938028 22376 tablet_service.cc:1467] Processing CreateTablet for tablet 1c176c5ed3794ee19d6936842c91cd9b (DEFAULT_TABLE table=test-table [id=dbc1055e3137460cad36d80a5949af8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:52.938510 22562 tablet_service.cc:1467] Processing CreateTablet for tablet 1c176c5ed3794ee19d6936842c91cd9b (DEFAULT_TABLE table=test-table [id=dbc1055e3137460cad36d80a5949af8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:52.939406 22376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c176c5ed3794ee19d6936842c91cd9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:52.939613 22562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c176c5ed3794ee19d6936842c91cd9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:52.940666 22191 tablet_service.cc:1467] Processing CreateTablet for tablet 1c176c5ed3794ee19d6936842c91cd9b (DEFAULT_TABLE table=test-table [id=dbc1055e3137460cad36d80a5949af8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:52.941557 22191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c176c5ed3794ee19d6936842c91cd9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:52.947332 22693 tablet_bootstrap.cc:492] T 1c176c5ed3794ee19d6936842c91cd9b P 072a42a89b4845f0956a593c0b889ef7: Bootstrap starting.
I20260807 15:09:52.947830 22690 tablet_bootstrap.cc:492] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9: Bootstrap starting.
I20260807 15:09:52.948815 22695 tablet_bootstrap.cc:492] T 1c176c5ed3794ee19d6936842c91cd9b P 1a8482df93db4aff9971b525cbae193e: Bootstrap starting.
I20260807 15:09:52.950201 22693 tablet_bootstrap.cc:654] T 1c176c5ed3794ee19d6936842c91cd9b P 072a42a89b4845f0956a593c0b889ef7: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:52.950632 22690 tablet_bootstrap.cc:654] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:52.950883 22695 tablet_bootstrap.cc:654] T 1c176c5ed3794ee19d6936842c91cd9b P 1a8482df93db4aff9971b525cbae193e: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:52.951445 22693 log.cc:824] T 1c176c5ed3794ee19d6936842c91cd9b P 072a42a89b4845f0956a593c0b889ef7: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:52.951812 22695 log.cc:824] T 1c176c5ed3794ee19d6936842c91cd9b P 1a8482df93db4aff9971b525cbae193e: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:52.951932 22690 log.cc:824] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:52.953711 22693 tablet_bootstrap.cc:492] T 1c176c5ed3794ee19d6936842c91cd9b P 072a42a89b4845f0956a593c0b889ef7: No bootstrap required, opened a new log
I20260807 15:09:52.953940 22695 tablet_bootstrap.cc:492] T 1c176c5ed3794ee19d6936842c91cd9b P 1a8482df93db4aff9971b525cbae193e: No bootstrap required, opened a new log
I20260807 15:09:52.953940 22690 tablet_bootstrap.cc:492] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9: No bootstrap required, opened a new log
I20260807 15:09:52.953958 22693 ts_tablet_manager.cc:1397] T 1c176c5ed3794ee19d6936842c91cd9b P 072a42a89b4845f0956a593c0b889ef7: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 15:09:52.954134 22690 ts_tablet_manager.cc:1397] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260807 15:09:52.954134 22695 ts_tablet_manager.cc:1397] T 1c176c5ed3794ee19d6936842c91cd9b P 1a8482df93db4aff9971b525cbae193e: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260807 15:09:52.957139 22693 raft_consensus.cc:348] T 1c176c5ed3794ee19d6936842c91cd9b P 072a42a89b4845f0956a593c0b889ef7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a8482df93db4aff9971b525cbae193e" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44213 } } peers { permanent_uuid: "0220f6ef067f4d26a0dd7cdd76dbbef9" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 34089 } } peers { permanent_uuid: "072a42a89b4845f0956a593c0b889ef7" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43853 } }
I20260807 15:09:52.957485 22693 raft_consensus.cc:374] T 1c176c5ed3794ee19d6936842c91cd9b P 072a42a89b4845f0956a593c0b889ef7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:52.957543 22693 raft_consensus.cc:729] T 1c176c5ed3794ee19d6936842c91cd9b P 072a42a89b4845f0956a593c0b889ef7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 072a42a89b4845f0956a593c0b889ef7, State: Initialized, Role: FOLLOWER
I20260807 15:09:52.958009 22693 consensus_queue.cc:260] T 1c176c5ed3794ee19d6936842c91cd9b P 072a42a89b4845f0956a593c0b889ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a8482df93db4aff9971b525cbae193e" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44213 } } peers { permanent_uuid: "0220f6ef067f4d26a0dd7cdd76dbbef9" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 34089 } } peers { permanent_uuid: "072a42a89b4845f0956a593c0b889ef7" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43853 } }
I20260807 15:09:52.958331 22690 raft_consensus.cc:348] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a8482df93db4aff9971b525cbae193e" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44213 } } peers { permanent_uuid: "0220f6ef067f4d26a0dd7cdd76dbbef9" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 34089 } } peers { permanent_uuid: "072a42a89b4845f0956a593c0b889ef7" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43853 } }
I20260807 15:09:52.958626 22690 raft_consensus.cc:374] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:52.958923 22667 heartbeater.cc:503] Master 127.15.226.126:45741 was elected leader, sending a full tablet report...
I20260807 15:09:52.959175 22693 ts_tablet_manager.cc:1428] T 1c176c5ed3794ee19d6936842c91cd9b P 072a42a89b4845f0956a593c0b889ef7: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 15:09:52.958711 22690 raft_consensus.cc:729] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0220f6ef067f4d26a0dd7cdd76dbbef9, State: Initialized, Role: FOLLOWER
I20260807 15:09:52.959527 22695 raft_consensus.cc:348] T 1c176c5ed3794ee19d6936842c91cd9b P 1a8482df93db4aff9971b525cbae193e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a8482df93db4aff9971b525cbae193e" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44213 } } peers { permanent_uuid: "0220f6ef067f4d26a0dd7cdd76dbbef9" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 34089 } } peers { permanent_uuid: "072a42a89b4845f0956a593c0b889ef7" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43853 } }
I20260807 15:09:52.959786 22695 raft_consensus.cc:374] T 1c176c5ed3794ee19d6936842c91cd9b P 1a8482df93db4aff9971b525cbae193e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:52.959846 22695 raft_consensus.cc:729] T 1c176c5ed3794ee19d6936842c91cd9b P 1a8482df93db4aff9971b525cbae193e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a8482df93db4aff9971b525cbae193e, State: Initialized, Role: FOLLOWER
I20260807 15:09:52.959782 22690 consensus_queue.cc:260] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a8482df93db4aff9971b525cbae193e" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44213 } } peers { permanent_uuid: "0220f6ef067f4d26a0dd7cdd76dbbef9" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 34089 } } peers { permanent_uuid: "072a42a89b4845f0956a593c0b889ef7" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43853 } }
I20260807 15:09:52.960362 22695 consensus_queue.cc:260] T 1c176c5ed3794ee19d6936842c91cd9b P 1a8482df93db4aff9971b525cbae193e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a8482df93db4aff9971b525cbae193e" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44213 } } peers { permanent_uuid: "0220f6ef067f4d26a0dd7cdd76dbbef9" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 34089 } } peers { permanent_uuid: "072a42a89b4845f0956a593c0b889ef7" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43853 } }
I20260807 15:09:52.960624 22463 heartbeater.cc:503] Master 127.15.226.126:45741 was elected leader, sending a full tablet report...
I20260807 15:09:52.960923 22690 ts_tablet_manager.cc:1428] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9: Time spent starting tablet: real 0.007s user 0.005s sys 0.002s
I20260807 15:09:52.961426 22695 ts_tablet_manager.cc:1428] T 1c176c5ed3794ee19d6936842c91cd9b P 1a8482df93db4aff9971b525cbae193e: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260807 15:09:52.961819 22279 heartbeater.cc:503] Master 127.15.226.126:45741 was elected leader, sending a full tablet report...
I20260807 15:09:53.098547 22701 raft_consensus.cc:482] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:09:53.098903 22701 raft_consensus.cc:504] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a8482df93db4aff9971b525cbae193e" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44213 } } peers { permanent_uuid: "0220f6ef067f4d26a0dd7cdd76dbbef9" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 34089 } } peers { permanent_uuid: "072a42a89b4845f0956a593c0b889ef7" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43853 } }
W20260807 15:09:53.099117 22280 tablet.cc:2369] T 1c176c5ed3794ee19d6936842c91cd9b P 1a8482df93db4aff9971b525cbae193e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:09:53.100382 22701 leader_election.cc:288] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a8482df93db4aff9971b525cbae193e (127.15.226.65:44213), 072a42a89b4845f0956a593c0b889ef7 (127.15.226.67:43853)
I20260807 15:09:53.104370 22685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:53.100793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:43743 (local address 127.15.226.65:44213)
0807 15:09:53.100910 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:53.100917 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:53.101094 (+ 177us) server_negotiation.cc:407] Connection header received
0807 15:09:53.101187 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:53.101202 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:53.101245 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:53.101337 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:53.101643 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:53.102165 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:53.103342 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:53.103771 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:53.103910 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:53.103968 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:53.104166 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 15:09:53.104249 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 15:09:53.104370 22465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:53.100652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:44213 (local address 127.15.226.66:43743)
0807 15:09:53.100905 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:53.100921 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:53.101082 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:53.101356 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:53.101359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:53.101378 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:53.101522 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:53.101528 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:53.102301 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:53.102305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:53.103208 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:53.103219 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:53.103910 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:53.103914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:53.104025 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:53.104045 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 15:09:53.104135 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 15:09:53.104220 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165}
I20260807 15:09:53.104492 22689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:53.100986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:36353 (local address 127.15.226.67:43853)
0807 15:09:53.101141 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:53.101147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:53.101245 (+ 98us) server_negotiation.cc:407] Connection header received
0807 15:09:53.101424 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:53.101431 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:53.101474 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:53.101566 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:09:53.101854 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:53.102344 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:53.103486 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:53.103890 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:53.104023 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:53.104076 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:53.104266 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 15:09:53.104341 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 15:09:53.104552 22705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:53.100736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:43853 (local address 127.15.226.66:36353)
0807 15:09:53.101132 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:53.101148 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:53.101307 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:53.101577 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:53.101581 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:53.101598 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:09:53.101729 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:53.101735 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:53.102468 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:53.102472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:53.103346 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:53.103356 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:53.104017 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:53.104021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:53.104127 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:53.104150 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 15:09:53.104249 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 15:09:53.104328 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":90,"threads_started":1}
I20260807 15:09:53.105238 22221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c176c5ed3794ee19d6936842c91cd9b" candidate_uuid: "0220f6ef067f4d26a0dd7cdd76dbbef9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a8482df93db4aff9971b525cbae193e" is_pre_election: true
I20260807 15:09:53.105221 22596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c176c5ed3794ee19d6936842c91cd9b" candidate_uuid: "0220f6ef067f4d26a0dd7cdd76dbbef9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "072a42a89b4845f0956a593c0b889ef7" is_pre_election: true
I20260807 15:09:53.105585 22596 raft_consensus.cc:2454] T 1c176c5ed3794ee19d6936842c91cd9b P 072a42a89b4845f0956a593c0b889ef7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0220f6ef067f4d26a0dd7cdd76dbbef9 in term 0.
I20260807 15:09:53.105585 22221 raft_consensus.cc:2454] T 1c176c5ed3794ee19d6936842c91cd9b P 1a8482df93db4aff9971b525cbae193e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0220f6ef067f4d26a0dd7cdd76dbbef9 in term 0.
I20260807 15:09:53.106179 22308 leader_election.cc:302] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0220f6ef067f4d26a0dd7cdd76dbbef9, 072a42a89b4845f0956a593c0b889ef7, 1a8482df93db4aff9971b525cbae193e; no voters:
I20260807 15:09:53.106532 22701 raft_consensus.cc:2781] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260807 15:09:53.106608 22464 tablet.cc:2369] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:09:53.106635 22701 raft_consensus.cc:482] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:09:53.106724 22701 raft_consensus.cc:3037] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:53.107899 22701 raft_consensus.cc:504] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a8482df93db4aff9971b525cbae193e" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44213 } } peers { permanent_uuid: "0220f6ef067f4d26a0dd7cdd76dbbef9" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 34089 } } peers { permanent_uuid: "072a42a89b4845f0956a593c0b889ef7" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43853 } }
I20260807 15:09:53.108296 22701 leader_election.cc:288] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9 [CANDIDATE]: Term 1 election: Requested vote from peers 1a8482df93db4aff9971b525cbae193e (127.15.226.65:44213), 072a42a89b4845f0956a593c0b889ef7 (127.15.226.67:43853)
I20260807 15:09:53.108731 22596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c176c5ed3794ee19d6936842c91cd9b" candidate_uuid: "0220f6ef067f4d26a0dd7cdd76dbbef9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "072a42a89b4845f0956a593c0b889ef7"
I20260807 15:09:53.108826 22221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c176c5ed3794ee19d6936842c91cd9b" candidate_uuid: "0220f6ef067f4d26a0dd7cdd76dbbef9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a8482df93db4aff9971b525cbae193e"
I20260807 15:09:53.108886 22596 raft_consensus.cc:3037] T 1c176c5ed3794ee19d6936842c91cd9b P 072a42a89b4845f0956a593c0b889ef7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:53.109021 22221 raft_consensus.cc:3037] T 1c176c5ed3794ee19d6936842c91cd9b P 1a8482df93db4aff9971b525cbae193e [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:53.110093 22596 raft_consensus.cc:2454] T 1c176c5ed3794ee19d6936842c91cd9b P 072a42a89b4845f0956a593c0b889ef7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0220f6ef067f4d26a0dd7cdd76dbbef9 in term 1.
I20260807 15:09:53.110162 22221 raft_consensus.cc:2454] T 1c176c5ed3794ee19d6936842c91cd9b P 1a8482df93db4aff9971b525cbae193e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0220f6ef067f4d26a0dd7cdd76dbbef9 in term 1.
I20260807 15:09:53.110487 22307 leader_election.cc:302] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0220f6ef067f4d26a0dd7cdd76dbbef9, 072a42a89b4845f0956a593c0b889ef7; no voters:
I20260807 15:09:53.110737 22701 raft_consensus.cc:2781] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:53.111012 22701 raft_consensus.cc:686] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9 [term 1 LEADER]: Becoming Leader. State: Replica: 0220f6ef067f4d26a0dd7cdd76dbbef9, State: Running, Role: LEADER
I20260807 15:09:53.111392 22701 consensus_queue.cc:237] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a8482df93db4aff9971b525cbae193e" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44213 } } peers { permanent_uuid: "0220f6ef067f4d26a0dd7cdd76dbbef9" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 34089 } } peers { permanent_uuid: "072a42a89b4845f0956a593c0b889ef7" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43853 } }
I20260807 15:09:53.114560 22012 catalog_manager.cc:5495] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0220f6ef067f4d26a0dd7cdd76dbbef9 (127.15.226.66). New cstate: current_term: 1 leader_uuid: "0220f6ef067f4d26a0dd7cdd76dbbef9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a8482df93db4aff9971b525cbae193e" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0220f6ef067f4d26a0dd7cdd76dbbef9" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 34089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "072a42a89b4845f0956a593c0b889ef7" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43853 } health_report { overall_health: UNKNOWN } } }
W20260807 15:09:53.117646 22668 tablet.cc:2369] T 1c176c5ed3794ee19d6936842c91cd9b P 072a42a89b4845f0956a593c0b889ef7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:09:53.172403 22686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:53.168474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46922 (local address 127.15.226.66:34089)
0807 15:09:53.168616 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:53.168619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:53.168633 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:09:53.168724 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:53.168727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:53.168784 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:53.168903 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:09:53.169234 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:53.169823 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:53.170868 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:53.171077 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:53.171158 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:53.171521 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:09:53.172068 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:09:53.172166 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:53.172233 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 15:09:53.172284 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 15:09:53.178927 22596 raft_consensus.cc:1261] T 1c176c5ed3794ee19d6936842c91cd9b P 072a42a89b4845f0956a593c0b889ef7 [term 1 FOLLOWER]: Refusing update from remote peer 0220f6ef067f4d26a0dd7cdd76dbbef9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:53.178956 22221 raft_consensus.cc:1261] T 1c176c5ed3794ee19d6936842c91cd9b P 1a8482df93db4aff9971b525cbae193e [term 1 FOLLOWER]: Refusing update from remote peer 0220f6ef067f4d26a0dd7cdd76dbbef9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:53.179795 22701 consensus_queue.cc:1035] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a8482df93db4aff9971b525cbae193e" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:53.180073 22708 consensus_queue.cc:1035] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "072a42a89b4845f0956a593c0b889ef7" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:53.188200 22712 mvcc.cc:205] Tried to move back new op lower bound from 7315928650452131840 to 7315928650187579392. Current Snapshot: MvccSnapshot[applied={T|T < 7315928650452131840}]
I20260807 15:09:53.189989 22713 mvcc.cc:205] Tried to move back new op lower bound from 7315928650452131840 to 7315928650187579392. Current Snapshot: MvccSnapshot[applied={T|T < 7315928650452131840}]
I20260807 15:09:53.190306 22714 mvcc.cc:205] Tried to move back new op lower bound from 7315928650452131840 to 7315928650187579392. Current Snapshot: MvccSnapshot[applied={T|T < 7315928650452131840}]
I20260807 15:09:53.198194 22014 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:53.198431 22014 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:53.201517 22014 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5be38c90c754b24abe2f3fb4b2be231: Sending DeleteTablet for 3 replicas of tablet 1c176c5ed3794ee19d6936842c91cd9b
I20260807 15:09:53.202606 22562 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c176c5ed3794ee19d6936842c91cd9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:53 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:60462
I20260807 15:09:53.202605 22191 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c176c5ed3794ee19d6936842c91cd9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:53 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:35190
I20260807 15:09:53.203301 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 22100
I20260807 15:09:53.203274 22376 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c176c5ed3794ee19d6936842c91cd9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:53 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:46910
I20260807 15:09:53.207314 22729 tablet_replica.cc:330] T 1c176c5ed3794ee19d6936842c91cd9b P 072a42a89b4845f0956a593c0b889ef7: stopping tablet replica
I20260807 15:09:53.207803 22729 raft_consensus.cc:2229] T 1c176c5ed3794ee19d6936842c91cd9b P 072a42a89b4845f0956a593c0b889ef7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:53.208277 22729 raft_consensus.cc:2258] T 1c176c5ed3794ee19d6936842c91cd9b P 072a42a89b4845f0956a593c0b889ef7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:53.210175 22729 ts_tablet_manager.cc:1905] T 1c176c5ed3794ee19d6936842c91cd9b P 072a42a89b4845f0956a593c0b889ef7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:53.210484 22730 tablet_replica.cc:330] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9: stopping tablet replica
I20260807 15:09:53.210848 22730 raft_consensus.cc:2229] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:09:53.211876 22730 raft_consensus.cc:2258] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:53.213352 22729 ts_tablet_manager.cc:1918] T 1c176c5ed3794ee19d6936842c91cd9b P 072a42a89b4845f0956a593c0b889ef7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:09:53.213461 22729 log.cc:1196] T 1c176c5ed3794ee19d6936842c91cd9b P 072a42a89b4845f0956a593c0b889ef7: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/1c176c5ed3794ee19d6936842c91cd9b
I20260807 15:09:53.213585 22730 ts_tablet_manager.cc:1905] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:53.213840 22729 ts_tablet_manager.cc:1939] T 1c176c5ed3794ee19d6936842c91cd9b P 072a42a89b4845f0956a593c0b889ef7: Deleting consensus metadata
I20260807 15:09:53.215342 21994 catalog_manager.cc:4841] TS 072a42a89b4845f0956a593c0b889ef7 (127.15.226.67:43853): tablet 1c176c5ed3794ee19d6936842c91cd9b (table test-table [id=dbc1055e3137460cad36d80a5949af8c]) successfully deleted
I20260807 15:09:53.216305 22730 ts_tablet_manager.cc:1918] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:09:53.216399 22730 log.cc:1196] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/1c176c5ed3794ee19d6936842c91cd9b
I20260807 15:09:53.216642 22730 ts_tablet_manager.cc:1939] T 1c176c5ed3794ee19d6936842c91cd9b P 0220f6ef067f4d26a0dd7cdd76dbbef9: Deleting consensus metadata
I20260807 15:09:53.217705 21994 catalog_manager.cc:4841] TS 0220f6ef067f4d26a0dd7cdd76dbbef9 (127.15.226.66:34089): tablet 1c176c5ed3794ee19d6936842c91cd9b (table test-table [id=dbc1055e3137460cad36d80a5949af8c]) successfully deleted
W20260807 15:09:53.220108 21995 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.226.65:44213 (error 108)
I20260807 15:09:53.221104 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 22284
I20260807 15:09:53.221349 22683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:53.221006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:44213 (local address 127.0.0.1:35194)
0807 15:09:53.221176 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:53.221260 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.65:44213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260807 15:09:53.228117 21995 catalog_manager.cc:4584] TS 1a8482df93db4aff9971b525cbae193e (127.15.226.65:44213): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1c176c5ed3794ee19d6936842c91cd9b: Network error: Client connection negotiation failed: client connection to 127.15.226.65:44213: connect: Connection refused (error 111)
I20260807 15:09:53.236778 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 22468
I20260807 15:09:53.252461 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 21976
2026-08-07T15:09:53Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2812 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T15:09:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:09:53Z Disabled control of system clock
I20260807 15:09:53.291618 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:34177
--webserver_interface=127.15.226.126
--webserver_port=0
--builtin_ntp_servers=127.15.226.84:39027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:34177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 15:09:53.412453 22738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:53.412774 22738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:53.412815 22738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:53.416989 22738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:53.417079 22738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:53.417095 22738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:53.417114 22738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:53.417128 22738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:09:53.422211 22738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:39027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:34177
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:34177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:53.423491 22738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:53.425057 22738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:53.434803 22743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:53.434907 22744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:53.435066 22738 server_base.cc:1034] running on GCE node
W20260807 15:09:53.434875 22747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:53.435839 22738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:53.437093 22738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:53.438285 22738 hybrid_clock.cc:648] HybridClock initialized: now 1786115393438259 us; error 49 us; skew 500 ppm
I20260807 15:09:53.440557 22738 webserver.cc:460] Webserver started at http://127.15.226.126:36797/ using document root <none> and password file <none>
I20260807 15:09:53.441198 22738 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:53.441280 22738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:53.441529 22738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:53.443930 22738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "6b5358b6b752439c80f39cab548c8ff4"
format_stamp: "Formatted at 2026-08-07 15:09:53 on dist-test-slave-3kk6"
server_key: "75e60805daa55716dc27ac942216372f"
server_key_iv: "1e59e2c503c72ec45bd8d62c91717188"
server_key_version: "encryptionkey@0"
I20260807 15:09:53.444597 22738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "6b5358b6b752439c80f39cab548c8ff4"
format_stamp: "Formatted at 2026-08-07 15:09:53 on dist-test-slave-3kk6"
server_key: "75e60805daa55716dc27ac942216372f"
server_key_iv: "1e59e2c503c72ec45bd8d62c91717188"
server_key_version: "encryptionkey@0"
I20260807 15:09:53.448909 22738 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:09:53.451714 22755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:53.453135 22738 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:09:53.453302 22738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "6b5358b6b752439c80f39cab548c8ff4"
format_stamp: "Formatted at 2026-08-07 15:09:53 on dist-test-slave-3kk6"
server_key: "75e60805daa55716dc27ac942216372f"
server_key_iv: "1e59e2c503c72ec45bd8d62c91717188"
server_key_version: "encryptionkey@0"
I20260807 15:09:53.453452 22738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:53.458626 22738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:53.459376 22738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:53.459589 22738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:53.466822 22738 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:34177
I20260807 15:09:53.466907 22827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:34177 every 8 connection(s)
I20260807 15:09:53.467918 22738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:09:53.468716 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 22738
I20260807 15:09:53.468850 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:09:53.469146 16265 external_mini_cluster.cc:1428] Setting key 5fcc222ff08f7d3cf60d86be083c1d05
I20260807 15:09:53.471611 22829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:53.476405 22832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:53.470593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57736 (local address 127.15.226.126:34177)
0807 15:09:53.471999 (+ 1406us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:53.472022 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:53.472069 (+ 47us) server_negotiation.cc:407] Connection header received
0807 15:09:53.472997 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:53.473019 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:53.473491 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:53.473886 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:53.474770 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:53.474777 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:53.474794 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:09:53.474808 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:53.474881 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:09:53.474907 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:53.475068 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:53.475145 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 15:09:53.475415 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":954,"thread_start_us":719,"threads_started":1}
I20260807 15:09:53.479118 22829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4: Bootstrap starting.
I20260807 15:09:53.481384 22829 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:53.482136 22829 log.cc:824] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:53.484855 22829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4: No bootstrap required, opened a new log
I20260807 15:09:53.488756 22829 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b5358b6b752439c80f39cab548c8ff4" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34177 } }
I20260807 15:09:53.489018 22829 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:53.489096 22829 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b5358b6b752439c80f39cab548c8ff4, State: Initialized, Role: FOLLOWER
I20260807 15:09:53.489583 22829 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b5358b6b752439c80f39cab548c8ff4" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34177 } }
I20260807 15:09:53.489758 22829 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:09:53.489820 22829 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:09:53.489929 22829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:53.491147 22829 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b5358b6b752439c80f39cab548c8ff4" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34177 } }
I20260807 15:09:53.491513 22829 leader_election.cc:302] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b5358b6b752439c80f39cab548c8ff4; no voters:
I20260807 15:09:53.491910 22829 leader_election.cc:288] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:09:53.492071 22834 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:53.492332 22834 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4 [term 1 LEADER]: Becoming Leader. State: Replica: 6b5358b6b752439c80f39cab548c8ff4, State: Running, Role: LEADER
I20260807 15:09:53.492653 22834 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b5358b6b752439c80f39cab548c8ff4" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34177 } }
I20260807 15:09:53.493250 22829 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:53.494534 22835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b5358b6b752439c80f39cab548c8ff4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b5358b6b752439c80f39cab548c8ff4" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34177 } } }
I20260807 15:09:53.494704 22835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:53.495026 22836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b5358b6b752439c80f39cab548c8ff4. Latest consensus state: current_term: 1 leader_uuid: "6b5358b6b752439c80f39cab548c8ff4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b5358b6b752439c80f39cab548c8ff4" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34177 } } }
I20260807 15:09:53.495122 22836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4 [sys.catalog]: This master's current role is: LEADER
W20260807 15:09:53.498224 22854 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:09:53.498337 22854 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:09:53.498414 22846 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:09:53.499325 22846 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:09:53.505090 22846 catalog_manager.cc:1347] Generated new cluster ID: 0323b957de744d64b9aa7197dce4dc71
I20260807 15:09:53.505188 22846 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:09:53.517220 22846 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:09:53.517592 22846 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:09:53.532132 22846 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4: Generated new TSK 0
I20260807 15:09:53.533097 22846 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:09:53.545429 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34177
--builtin_ntp_servers=127.15.226.84:39027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 15:09:53.668205 22858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:53.668520 22858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:53.668563 22858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:53.672837 22858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:53.672923 22858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:53.673034 22858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:09:53.678009 22858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:39027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:53.679180 22858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:53.680613 22858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:09:53.690986 22858 server_base.cc:1034] running on GCE node
W20260807 15:09:53.690928 22870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:53.690928 22868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:53.690934 22867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:53.691615 22858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:53.692262 22858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:53.693421 22858 hybrid_clock.cc:648] HybridClock initialized: now 1786115393693408 us; error 30 us; skew 500 ppm
I20260807 15:09:53.695603 22858 webserver.cc:460] Webserver started at http://127.15.226.65:38695/ using document root <none> and password file <none>
I20260807 15:09:53.696211 22858 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:53.696275 22858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:53.696498 22858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:53.698729 22858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "f6a250676d964318914683e10b1c80d8"
format_stamp: "Formatted at 2026-08-07 15:09:53 on dist-test-slave-3kk6"
server_key: "42aee8fdcd5c375e08b2a5429681385b"
server_key_iv: "50e1ef2eb2d501e2ec13b928679a3be8"
server_key_version: "encryptionkey@0"
I20260807 15:09:53.699376 22858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "f6a250676d964318914683e10b1c80d8"
format_stamp: "Formatted at 2026-08-07 15:09:53 on dist-test-slave-3kk6"
server_key: "42aee8fdcd5c375e08b2a5429681385b"
server_key_iv: "50e1ef2eb2d501e2ec13b928679a3be8"
server_key_version: "encryptionkey@0"
I20260807 15:09:53.703384 22858 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:09:53.706000 22878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:53.707456 22858 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 15:09:53.707599 22858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "f6a250676d964318914683e10b1c80d8"
format_stamp: "Formatted at 2026-08-07 15:09:53 on dist-test-slave-3kk6"
server_key: "42aee8fdcd5c375e08b2a5429681385b"
server_key_iv: "50e1ef2eb2d501e2ec13b928679a3be8"
server_key_version: "encryptionkey@0"
I20260807 15:09:53.707731 22858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:53.712806 22858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:53.713491 22858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:53.713685 22858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:53.714249 22858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:53.715274 22858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:53.715344 22858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:53.715421 22858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:53.715464 22858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:53.724663 22858 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:44839
I20260807 15:09:53.724687 23048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:44839 every 8 connection(s)
I20260807 15:09:53.725605 22858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:09:53.732026 22832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:53.727586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:58519 (local address 127.15.226.126:34177)
0807 15:09:53.727736 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:53.727740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:53.728772 (+ 1032us) server_negotiation.cc:407] Connection header received
0807 15:09:53.730173 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:53.730177 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:53.730229 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:53.730353 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:53.731499 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:53.731503 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:53.731506 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:09:53.731520 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:53.731542 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:09:53.731558 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:53.731662 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:53.731834 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 15:09:53.731883 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 15:09:53.732955 23053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:53.727935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34177 (local address 127.15.226.65:58519)
0807 15:09:53.728607 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:53.728649 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:53.729941 (+ 1292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:53.730523 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:53.730530 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:53.730618 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:53.731233 (+ 615us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:09:53.731234 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:53.731275 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:09:53.731347 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:53.731654 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:53.731664 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:53.731671 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 15:09:53.731889 (+ 218us) client_negotiation.cc:240] Negotiation successful
0807 15:09:53.732144 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":121,"threads_started":1}
I20260807 15:09:53.733737 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 22858
I20260807 15:09:53.733882 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:09:53.734180 16265 external_mini_cluster.cc:1428] Setting key 6884c2d7e7761d7422988f68bcab1271
I20260807 15:09:53.734270 23051 heartbeater.cc:347] Connected to a master server at 127.15.226.126:34177
I20260807 15:09:53.734545 23051 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:53.735146 23051 heartbeater.cc:511] Master 127.15.226.126:34177 requested a full tablet report, sending...
I20260807 15:09:53.736336 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34177
--builtin_ntp_servers=127.15.226.84:39027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 15:09:53.736807 22779 ts_manager.cc:194] Registered new tserver with Master: f6a250676d964318914683e10b1c80d8 (127.15.226.65:44839)
W20260807 15:09:53.857611 23054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:53.857970 23054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:53.858039 23054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:53.862263 23054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:53.862356 23054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:53.862418 23054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:09:53.867525 23054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:39027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:53.868710 23054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:53.870131 23054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:53.880781 23060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:53.880996 23054 server_base.cc:1034] running on GCE node
W20260807 15:09:53.880664 23059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:53.880656 23064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:53.881572 23054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:53.882299 23054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:53.883502 23054 hybrid_clock.cc:648] HybridClock initialized: now 1786115393883472 us; error 43 us; skew 500 ppm
I20260807 15:09:53.885561 23054 webserver.cc:460] Webserver started at http://127.15.226.66:37917/ using document root <none> and password file <none>
I20260807 15:09:53.886193 23054 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:53.886293 23054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:53.886576 23054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:53.888901 23054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "006e74f910264e5598ec3da6508b29f5"
format_stamp: "Formatted at 2026-08-07 15:09:53 on dist-test-slave-3kk6"
server_key: "ad5fddb99139185f568a9893c3cc0446"
server_key_iv: "2751fed1ee4a972e046493eebf64824a"
server_key_version: "encryptionkey@0"
I20260807 15:09:53.889652 23054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "006e74f910264e5598ec3da6508b29f5"
format_stamp: "Formatted at 2026-08-07 15:09:53 on dist-test-slave-3kk6"
server_key: "ad5fddb99139185f568a9893c3cc0446"
server_key_iv: "2751fed1ee4a972e046493eebf64824a"
server_key_version: "encryptionkey@0"
I20260807 15:09:53.894227 23054 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 15:09:53.897372 23074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:53.898809 23054 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:09:53.898965 23054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "006e74f910264e5598ec3da6508b29f5"
format_stamp: "Formatted at 2026-08-07 15:09:53 on dist-test-slave-3kk6"
server_key: "ad5fddb99139185f568a9893c3cc0446"
server_key_iv: "2751fed1ee4a972e046493eebf64824a"
server_key_version: "encryptionkey@0"
I20260807 15:09:53.899113 23054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:53.904367 23054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:53.905104 23054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:53.905321 23054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:53.905946 23054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:53.907022 23054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:53.907099 23054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:53.907187 23054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:53.907238 23054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:53.917454 23054 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:45937
I20260807 15:09:53.917477 23240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:45937 every 8 connection(s)
I20260807 15:09:53.918602 23054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:09:53.924640 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 23054
I20260807 15:09:53.924782 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:09:53.925153 16265 external_mini_cluster.cc:1428] Setting key 8775f793bb1332757ca0b2b9e9e62e6c
I20260807 15:09:53.925590 22832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:53.920760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:59529 (local address 127.15.226.126:34177)
0807 15:09:53.920956 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:53.920960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:53.922163 (+ 1203us) server_negotiation.cc:407] Connection header received
0807 15:09:53.923578 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:53.923581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:53.923622 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:53.923776 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:53.925096 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:53.925100 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:53.925102 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:09:53.925114 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:53.925136 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:09:53.925144 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:53.925252 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:53.925418 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 15:09:53.925460 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 15:09:53.926535 23244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:53.921203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34177 (local address 127.15.226.66:59529)
0807 15:09:53.921978 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:53.922025 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:53.923340 (+ 1315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:53.923959 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:53.923966 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:53.924067 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:53.924749 (+ 682us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:09:53.924754 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:53.924818 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:09:53.924916 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:53.925247 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:53.925258 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:53.925265 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 15:09:53.925454 (+ 189us) client_negotiation.cc:240] Negotiation successful
0807 15:09:53.925752 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":124,"threads_started":1}
I20260807 15:09:53.927480 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34177
--builtin_ntp_servers=127.15.226.84:39027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 15:09:53.927835 23241 heartbeater.cc:347] Connected to a master server at 127.15.226.126:34177
I20260807 15:09:53.928082 23241 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:53.928522 23241 heartbeater.cc:511] Master 127.15.226.126:34177 requested a full tablet report, sending...
I20260807 15:09:53.929729 22779 ts_manager.cc:194] Registered new tserver with Master: 006e74f910264e5598ec3da6508b29f5 (127.15.226.66:45937)
W20260807 15:09:54.048331 23246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:54.048677 23246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:54.048761 23246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:54.052928 23246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:54.053035 23246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:54.053129 23246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:09:54.058434 23246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:39027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:54.059695 23246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:54.061199 23246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:54.071719 23255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:54.071743 23254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:54.071719 23259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:54.072242 23246 server_base.cc:1034] running on GCE node
I20260807 15:09:54.072760 23246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:54.073518 23246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:54.074743 23246 hybrid_clock.cc:648] HybridClock initialized: now 1786115394074702 us; error 52 us; skew 500 ppm
I20260807 15:09:54.081205 23246 webserver.cc:460] Webserver started at http://127.15.226.67:34089/ using document root <none> and password file <none>
I20260807 15:09:54.081923 23246 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:54.082036 23246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:54.082298 23246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:54.084746 23246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "bd8fa3e16bdd49e0b2bc6d0c20387d93"
format_stamp: "Formatted at 2026-08-07 15:09:54 on dist-test-slave-3kk6"
server_key: "42010839738a99141d15b2744157c930"
server_key_iv: "c296e0d9cac5c40ca7741ec239d716af"
server_key_version: "encryptionkey@0"
I20260807 15:09:54.085412 23246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "bd8fa3e16bdd49e0b2bc6d0c20387d93"
format_stamp: "Formatted at 2026-08-07 15:09:54 on dist-test-slave-3kk6"
server_key: "42010839738a99141d15b2744157c930"
server_key_iv: "c296e0d9cac5c40ca7741ec239d716af"
server_key_version: "encryptionkey@0"
I20260807 15:09:54.089596 23246 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 15:09:54.092403 23266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:54.093775 23246 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:09:54.093922 23246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "bd8fa3e16bdd49e0b2bc6d0c20387d93"
format_stamp: "Formatted at 2026-08-07 15:09:54 on dist-test-slave-3kk6"
server_key: "42010839738a99141d15b2744157c930"
server_key_iv: "c296e0d9cac5c40ca7741ec239d716af"
server_key_version: "encryptionkey@0"
I20260807 15:09:54.094110 23246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:54.101615 23246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:54.102344 23246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:54.102623 23246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:54.103399 23246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:54.104543 23246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:54.104602 23246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:54.104652 23246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:54.104674 23246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:54.118386 23246 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:42497
I20260807 15:09:54.118470 23435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:42497 every 8 connection(s)
I20260807 15:09:54.119817 23246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:09:54.127319 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 23246
I20260807 15:09:54.127453 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:09:54.127758 16265 external_mini_cluster.cc:1428] Setting key 682b221359a0b33e373f985e6b7de31a
I20260807 15:09:54.128844 22832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:54.122583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:57175 (local address 127.15.226.126:34177)
0807 15:09:54.122771 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:54.122774 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:54.124290 (+ 1516us) server_negotiation.cc:407] Connection header received
0807 15:09:54.126545 (+ 2255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:54.126550 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:54.126598 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:54.126706 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:54.128268 (+ 1562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:54.128271 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:54.128274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:09:54.128290 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:54.128323 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:09:54.128334 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:54.128420 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:54.128637 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 15:09:54.128688 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 15:09:54.129812 23439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:54.123227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34177 (local address 127.15.226.67:57175)
0807 15:09:54.124088 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:54.124136 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:54.126211 (+ 2075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:54.127064 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:54.127075 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:54.127195 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:54.127941 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:09:54.127943 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:54.127997 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:09:54.128094 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:54.128464 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:54.128474 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:54.128480 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:09:54.128678 (+ 198us) client_negotiation.cc:240] Negotiation successful
0807 15:09:54.129029 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":182,"threads_started":1}
I20260807 15:09:54.131217 23437 heartbeater.cc:347] Connected to a master server at 127.15.226.126:34177
I20260807 15:09:54.131466 23437 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:54.132005 23437 heartbeater.cc:511] Master 127.15.226.126:34177 requested a full tablet report, sending...
I20260807 15:09:54.133203 22779 ts_manager.cc:194] Registered new tserver with Master: bd8fa3e16bdd49e0b2bc6d0c20387d93 (127.15.226.67:42497)
I20260807 15:09:54.143973 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:09:54.148581 22832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:54.146584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57750 (local address 127.15.226.126:34177)
0807 15:09:54.146772 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:54.146778 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:54.146936 (+ 158us) server_negotiation.cc:407] Connection header received
0807 15:09:54.147092 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:54.147095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:54.147143 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:54.147233 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:54.148160 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:54.148163 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:54.148166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:09:54.148178 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:54.148198 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:09:54.148206 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:54.148312 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:54.148431 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 15:09:54.148469 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 15:09:54.151968 22779 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57750:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:09:54.155211 22779 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:09:54.168143 23458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:54.164304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:42497 (local address 127.0.0.1:58332)
0807 15:09:54.165625 (+ 1321us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:54.165639 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:54.165724 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:54.166200 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:54.166208 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:54.166289 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:54.167337 (+ 1048us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:09:54.167339 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:54.167347 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:09:54.167371 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:54.167804 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:54.167807 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:54.167811 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:09:54.167889 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 15:09:54.167970 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1258,"mutex_wait_us":169,"thread_start_us":120,"threads_started":1}
I20260807 15:09:54.168375 23451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:54.164303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:44839 (local address 127.0.0.1:48268)
0807 15:09:54.165211 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:54.165266 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:54.165402 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:54.166202 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:54.166213 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:54.166287 (+ 74us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:54.167255 (+ 968us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:09:54.167257 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:54.167281 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:09:54.167372 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:54.167773 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:54.167783 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:54.167790 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 15:09:54.167891 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 15:09:54.167979 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"mutex_wait_us":3,"thread_start_us":108,"threads_started":1}
I20260807 15:09:54.168519 23463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:54.165234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46424 (local address 127.15.226.66:45937)
0807 15:09:54.166160 (+ 926us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:54.166172 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:54.166193 (+ 21us) server_negotiation.cc:407] Connection header received
0807 15:09:54.166242 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:54.166246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:54.166423 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:54.166876 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:54.167788 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:54.167794 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:54.167803 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:09:54.167814 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:54.167855 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:09:54.167883 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:54.168225 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:54.168287 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 15:09:54.168355 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":803,"thread_start_us":159,"threads_started":1}
I20260807 15:09:54.168146 23452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:54.165080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:45937 (local address 127.0.0.1:46424)
0807 15:09:54.165867 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:54.165878 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:54.165932 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:54.166591 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:54.166594 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:54.166615 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:54.167391 (+ 776us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:09:54.167393 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:54.167405 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:09:54.167436 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:54.167984 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:54.167987 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:54.167990 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:09:54.168042 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 15:09:54.168068 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"thread_start_us":116,"threads_started":1}
I20260807 15:09:54.168951 23453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:54.164467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48268 (local address 127.15.226.65:44839)
0807 15:09:54.165834 (+ 1367us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:54.165846 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:54.165867 (+ 21us) server_negotiation.cc:407] Connection header received
0807 15:09:54.165914 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:54.165919 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:54.166045 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:54.166631 (+ 586us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:54.167538 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:54.167542 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:54.167551 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:09:54.167562 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:54.167600 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:09:54.167623 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:54.168082 (+ 459us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:54.168136 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 15:09:54.168786 (+ 650us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1247,"thread_start_us":148,"threads_started":1}
I20260807 15:09:54.169080 23455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:54.164573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58332 (local address 127.15.226.67:42497)
0807 15:09:54.165549 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:54.165565 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:54.165739 (+ 174us) server_negotiation.cc:407] Connection header received
0807 15:09:54.165807 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:54.165813 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:54.166019 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:54.166180 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:54.167521 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:54.167529 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:54.167542 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:09:54.167554 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:54.167600 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:09:54.167664 (+ 64us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:54.168772 (+ 1108us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:54.168833 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 15:09:54.168903 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":858,"thread_start_us":165,"threads_started":1}
I20260807 15:09:54.171728 23339 tablet_service.cc:1467] Processing CreateTablet for tablet 816d0f94341e409f83854640b9a8a1f4 (DEFAULT_TABLE table=test-table [id=dc5c0148d90f4d82acedefbf48573438]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:54.171725 22949 tablet_service.cc:1467] Processing CreateTablet for tablet 816d0f94341e409f83854640b9a8a1f4 (DEFAULT_TABLE table=test-table [id=dc5c0148d90f4d82acedefbf48573438]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:54.171725 23148 tablet_service.cc:1467] Processing CreateTablet for tablet 816d0f94341e409f83854640b9a8a1f4 (DEFAULT_TABLE table=test-table [id=dc5c0148d90f4d82acedefbf48573438]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:09:54.172855 23339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 816d0f94341e409f83854640b9a8a1f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:54.172869 23148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 816d0f94341e409f83854640b9a8a1f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:54.172869 22949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 816d0f94341e409f83854640b9a8a1f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:54.179790 23464 tablet_bootstrap.cc:492] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93: Bootstrap starting.
I20260807 15:09:54.181216 23468 tablet_bootstrap.cc:492] T 816d0f94341e409f83854640b9a8a1f4 P f6a250676d964318914683e10b1c80d8: Bootstrap starting.
I20260807 15:09:54.181660 23470 tablet_bootstrap.cc:492] T 816d0f94341e409f83854640b9a8a1f4 P 006e74f910264e5598ec3da6508b29f5: Bootstrap starting.
I20260807 15:09:54.182875 23464 tablet_bootstrap.cc:654] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:54.183250 23468 tablet_bootstrap.cc:654] T 816d0f94341e409f83854640b9a8a1f4 P f6a250676d964318914683e10b1c80d8: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:54.183770 23470 tablet_bootstrap.cc:654] T 816d0f94341e409f83854640b9a8a1f4 P 006e74f910264e5598ec3da6508b29f5: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:54.183820 23464 log.cc:824] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:54.184001 23468 log.cc:824] T 816d0f94341e409f83854640b9a8a1f4 P f6a250676d964318914683e10b1c80d8: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:54.184648 23470 log.cc:824] T 816d0f94341e409f83854640b9a8a1f4 P 006e74f910264e5598ec3da6508b29f5: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:54.186084 23464 tablet_bootstrap.cc:492] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93: No bootstrap required, opened a new log
I20260807 15:09:54.186084 23468 tablet_bootstrap.cc:492] T 816d0f94341e409f83854640b9a8a1f4 P f6a250676d964318914683e10b1c80d8: No bootstrap required, opened a new log
I20260807 15:09:54.186316 23468 ts_tablet_manager.cc:1397] T 816d0f94341e409f83854640b9a8a1f4 P f6a250676d964318914683e10b1c80d8: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 15:09:54.186322 23464 ts_tablet_manager.cc:1397] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260807 15:09:54.186486 23470 tablet_bootstrap.cc:492] T 816d0f94341e409f83854640b9a8a1f4 P 006e74f910264e5598ec3da6508b29f5: No bootstrap required, opened a new log
I20260807 15:09:54.186688 23470 ts_tablet_manager.cc:1397] T 816d0f94341e409f83854640b9a8a1f4 P 006e74f910264e5598ec3da6508b29f5: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.001s
I20260807 15:09:54.190565 23470 raft_consensus.cc:348] T 816d0f94341e409f83854640b9a8a1f4 P 006e74f910264e5598ec3da6508b29f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a250676d964318914683e10b1c80d8" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44839 } } peers { permanent_uuid: "bd8fa3e16bdd49e0b2bc6d0c20387d93" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 42497 } } peers { permanent_uuid: "006e74f910264e5598ec3da6508b29f5" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 45937 } }
I20260807 15:09:54.190932 23470 raft_consensus.cc:374] T 816d0f94341e409f83854640b9a8a1f4 P 006e74f910264e5598ec3da6508b29f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:54.190995 23470 raft_consensus.cc:729] T 816d0f94341e409f83854640b9a8a1f4 P 006e74f910264e5598ec3da6508b29f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 006e74f910264e5598ec3da6508b29f5, State: Initialized, Role: FOLLOWER
I20260807 15:09:54.191169 23464 raft_consensus.cc:348] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a250676d964318914683e10b1c80d8" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44839 } } peers { permanent_uuid: "bd8fa3e16bdd49e0b2bc6d0c20387d93" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 42497 } } peers { permanent_uuid: "006e74f910264e5598ec3da6508b29f5" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 45937 } }
I20260807 15:09:54.191169 23468 raft_consensus.cc:348] T 816d0f94341e409f83854640b9a8a1f4 P f6a250676d964318914683e10b1c80d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a250676d964318914683e10b1c80d8" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44839 } } peers { permanent_uuid: "bd8fa3e16bdd49e0b2bc6d0c20387d93" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 42497 } } peers { permanent_uuid: "006e74f910264e5598ec3da6508b29f5" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 45937 } }
I20260807 15:09:54.191431 23464 raft_consensus.cc:374] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:54.191431 23468 raft_consensus.cc:374] T 816d0f94341e409f83854640b9a8a1f4 P f6a250676d964318914683e10b1c80d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:54.191494 23468 raft_consensus.cc:729] T 816d0f94341e409f83854640b9a8a1f4 P f6a250676d964318914683e10b1c80d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6a250676d964318914683e10b1c80d8, State: Initialized, Role: FOLLOWER
I20260807 15:09:54.191494 23464 raft_consensus.cc:729] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd8fa3e16bdd49e0b2bc6d0c20387d93, State: Initialized, Role: FOLLOWER
I20260807 15:09:54.191514 23470 consensus_queue.cc:260] T 816d0f94341e409f83854640b9a8a1f4 P 006e74f910264e5598ec3da6508b29f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a250676d964318914683e10b1c80d8" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44839 } } peers { permanent_uuid: "bd8fa3e16bdd49e0b2bc6d0c20387d93" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 42497 } } peers { permanent_uuid: "006e74f910264e5598ec3da6508b29f5" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 45937 } }
I20260807 15:09:54.191995 23468 consensus_queue.cc:260] T 816d0f94341e409f83854640b9a8a1f4 P f6a250676d964318914683e10b1c80d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a250676d964318914683e10b1c80d8" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44839 } } peers { permanent_uuid: "bd8fa3e16bdd49e0b2bc6d0c20387d93" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 42497 } } peers { permanent_uuid: "006e74f910264e5598ec3da6508b29f5" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 45937 } }
I20260807 15:09:54.191995 23464 consensus_queue.cc:260] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a250676d964318914683e10b1c80d8" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44839 } } peers { permanent_uuid: "bd8fa3e16bdd49e0b2bc6d0c20387d93" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 42497 } } peers { permanent_uuid: "006e74f910264e5598ec3da6508b29f5" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 45937 } }
I20260807 15:09:54.192402 23241 heartbeater.cc:503] Master 127.15.226.126:34177 was elected leader, sending a full tablet report...
I20260807 15:09:54.192934 23051 heartbeater.cc:503] Master 127.15.226.126:34177 was elected leader, sending a full tablet report...
I20260807 15:09:54.193063 23468 ts_tablet_manager.cc:1428] T 816d0f94341e409f83854640b9a8a1f4 P f6a250676d964318914683e10b1c80d8: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 15:09:54.193063 23464 ts_tablet_manager.cc:1428] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93: Time spent starting tablet: real 0.007s user 0.002s sys 0.006s
I20260807 15:09:54.193977 23470 ts_tablet_manager.cc:1428] T 816d0f94341e409f83854640b9a8a1f4 P 006e74f910264e5598ec3da6508b29f5: Time spent starting tablet: real 0.007s user 0.003s sys 0.002s
I20260807 15:09:54.194592 23437 heartbeater.cc:503] Master 127.15.226.126:34177 was elected leader, sending a full tablet report...
W20260807 15:09:54.226986 23052 tablet.cc:2369] T 816d0f94341e409f83854640b9a8a1f4 P f6a250676d964318914683e10b1c80d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:09:54.302800 23478 raft_consensus.cc:482] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:09:54.303076 23478 raft_consensus.cc:504] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a250676d964318914683e10b1c80d8" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44839 } } peers { permanent_uuid: "bd8fa3e16bdd49e0b2bc6d0c20387d93" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 42497 } } peers { permanent_uuid: "006e74f910264e5598ec3da6508b29f5" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 45937 } }
I20260807 15:09:54.304515 23478 leader_election.cc:288] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6a250676d964318914683e10b1c80d8 (127.15.226.65:44839), 006e74f910264e5598ec3da6508b29f5 (127.15.226.66:45937)
I20260807 15:09:54.306918 23439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:54.304800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:45937 (local address 127.15.226.67:45137)
0807 15:09:54.304947 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:54.304967 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:54.305076 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:54.305557 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:54.305561 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:54.305580 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:54.306163 (+ 583us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:09:54.306165 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:54.306174 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:09:54.306202 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:54.306608 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:54.306610 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:54.306612 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 15:09:54.306719 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 15:09:54.306761 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 15:09:54.307090 23453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:54.304985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:53885 (local address 127.15.226.65:44839)
0807 15:09:54.305148 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:54.305154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:54.305412 (+ 258us) server_negotiation.cc:407] Connection header received
0807 15:09:54.305489 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:54.305495 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:54.305531 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:54.305611 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:54.306590 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:54.306596 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:54.306600 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:09:54.306619 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:54.306648 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:09:54.306695 (+ 47us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:54.306814 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:54.306863 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 15:09:54.306913 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:09:54.307407 23480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:54.304800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:44839 (local address 127.15.226.67:53885)
0807 15:09:54.305316 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:54.305329 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:54.305407 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:54.305634 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:54.305637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:54.305655 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:54.306404 (+ 749us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:09:54.306405 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:54.306415 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:09:54.306438 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:54.306789 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:54.306792 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:54.306795 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 15:09:54.306860 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 15:09:54.306890 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":199,"thread_start_us":119,"threads_started":1}
I20260807 15:09:54.307509 23463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:54.305139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:45137 (local address 127.15.226.66:45937)
0807 15:09:54.305291 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:54.305295 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:54.305311 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:09:54.305355 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:54.305357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:54.305416 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:54.305507 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:54.306413 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:54.306417 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:54.306420 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:09:54.306435 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:54.306459 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:09:54.306473 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:54.306563 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:54.307321 (+ 758us) server_negotiation.cc:299] Negotiation successful
0807 15:09:54.307370 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:09:54.308095 22980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "816d0f94341e409f83854640b9a8a1f4" candidate_uuid: "bd8fa3e16bdd49e0b2bc6d0c20387d93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6a250676d964318914683e10b1c80d8" is_pre_election: true
I20260807 15:09:54.308131 23177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "816d0f94341e409f83854640b9a8a1f4" candidate_uuid: "bd8fa3e16bdd49e0b2bc6d0c20387d93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "006e74f910264e5598ec3da6508b29f5" is_pre_election: true
I20260807 15:09:54.308452 22980 raft_consensus.cc:2454] T 816d0f94341e409f83854640b9a8a1f4 P f6a250676d964318914683e10b1c80d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8fa3e16bdd49e0b2bc6d0c20387d93 in term 0.
I20260807 15:09:54.308501 23177 raft_consensus.cc:2454] T 816d0f94341e409f83854640b9a8a1f4 P 006e74f910264e5598ec3da6508b29f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd8fa3e16bdd49e0b2bc6d0c20387d93 in term 0.
I20260807 15:09:54.309023 23271 leader_election.cc:302] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 006e74f910264e5598ec3da6508b29f5, bd8fa3e16bdd49e0b2bc6d0c20387d93, f6a250676d964318914683e10b1c80d8; no voters:
I20260807 15:09:54.309309 23478 raft_consensus.cc:2781] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:09:54.309398 23478 raft_consensus.cc:482] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:09:54.309442 23478 raft_consensus.cc:3037] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:54.310561 23478 raft_consensus.cc:504] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a250676d964318914683e10b1c80d8" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44839 } } peers { permanent_uuid: "bd8fa3e16bdd49e0b2bc6d0c20387d93" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 42497 } } peers { permanent_uuid: "006e74f910264e5598ec3da6508b29f5" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 45937 } }
I20260807 15:09:54.311146 23478 leader_election.cc:288] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93 [CANDIDATE]: Term 1 election: Requested vote from peers f6a250676d964318914683e10b1c80d8 (127.15.226.65:44839), 006e74f910264e5598ec3da6508b29f5 (127.15.226.66:45937)
I20260807 15:09:54.311432 22980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "816d0f94341e409f83854640b9a8a1f4" candidate_uuid: "bd8fa3e16bdd49e0b2bc6d0c20387d93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6a250676d964318914683e10b1c80d8"
I20260807 15:09:54.311582 22980 raft_consensus.cc:3037] T 816d0f94341e409f83854640b9a8a1f4 P f6a250676d964318914683e10b1c80d8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:54.311555 23177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "816d0f94341e409f83854640b9a8a1f4" candidate_uuid: "bd8fa3e16bdd49e0b2bc6d0c20387d93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "006e74f910264e5598ec3da6508b29f5"
I20260807 15:09:54.311729 23177 raft_consensus.cc:3037] T 816d0f94341e409f83854640b9a8a1f4 P 006e74f910264e5598ec3da6508b29f5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:54.312847 22980 raft_consensus.cc:2454] T 816d0f94341e409f83854640b9a8a1f4 P f6a250676d964318914683e10b1c80d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8fa3e16bdd49e0b2bc6d0c20387d93 in term 1.
I20260807 15:09:54.313151 23271 leader_election.cc:302] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd8fa3e16bdd49e0b2bc6d0c20387d93, f6a250676d964318914683e10b1c80d8; no voters:
I20260807 15:09:54.313342 23478 raft_consensus.cc:2781] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:54.313565 23478 raft_consensus.cc:686] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93 [term 1 LEADER]: Becoming Leader. State: Replica: bd8fa3e16bdd49e0b2bc6d0c20387d93, State: Running, Role: LEADER
I20260807 15:09:54.313532 23177 raft_consensus.cc:2454] T 816d0f94341e409f83854640b9a8a1f4 P 006e74f910264e5598ec3da6508b29f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd8fa3e16bdd49e0b2bc6d0c20387d93 in term 1.
I20260807 15:09:54.313938 23478 consensus_queue.cc:237] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a250676d964318914683e10b1c80d8" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44839 } } peers { permanent_uuid: "bd8fa3e16bdd49e0b2bc6d0c20387d93" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 42497 } } peers { permanent_uuid: "006e74f910264e5598ec3da6508b29f5" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 45937 } }
I20260807 15:09:54.317755 22777 catalog_manager.cc:5495] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd8fa3e16bdd49e0b2bc6d0c20387d93 (127.15.226.67). New cstate: current_term: 1 leader_uuid: "bd8fa3e16bdd49e0b2bc6d0c20387d93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6a250676d964318914683e10b1c80d8" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd8fa3e16bdd49e0b2bc6d0c20387d93" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 42497 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "006e74f910264e5598ec3da6508b29f5" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 45937 } health_report { overall_health: UNKNOWN } } }
I20260807 15:09:54.366384 23455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:54.361602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58348 (local address 127.15.226.67:42497)
0807 15:09:54.361748 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:54.361752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:54.361766 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:09:54.361884 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:54.361886 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:54.361950 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:54.362025 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:54.365932 (+ 3907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:54.365935 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:54.365937 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:09:54.365953 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:54.365977 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:09:54.365986 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:54.366080 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:54.366163 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 15:09:54.366205 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 15:09:54.371153 23438 tablet.cc:2369] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:09:54.373147 23177 raft_consensus.cc:1261] T 816d0f94341e409f83854640b9a8a1f4 P 006e74f910264e5598ec3da6508b29f5 [term 1 FOLLOWER]: Refusing update from remote peer bd8fa3e16bdd49e0b2bc6d0c20387d93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:54.373211 22980 raft_consensus.cc:1261] T 816d0f94341e409f83854640b9a8a1f4 P f6a250676d964318914683e10b1c80d8 [term 1 FOLLOWER]: Refusing update from remote peer bd8fa3e16bdd49e0b2bc6d0c20387d93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:09:54.374046 23483 consensus_queue.cc:1035] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6a250676d964318914683e10b1c80d8" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:54.374305 23478 consensus_queue.cc:1035] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "006e74f910264e5598ec3da6508b29f5" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 45937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:09:54.392436 22778 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:54.392620 22778 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:09:54.395592 22778 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b5358b6b752439c80f39cab548c8ff4: Sending DeleteTablet for 3 replicas of tablet 816d0f94341e409f83854640b9a8a1f4
I20260807 15:09:54.396587 22949 tablet_service.cc:1514] Processing DeleteTablet for tablet 816d0f94341e409f83854640b9a8a1f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:54 UTC) from {username='slave'} at 127.0.0.1:48268
I20260807 15:09:54.396703 23148 tablet_service.cc:1514] Processing DeleteTablet for tablet 816d0f94341e409f83854640b9a8a1f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:54 UTC) from {username='slave'} at 127.0.0.1:46424
I20260807 15:09:54.396863 23339 tablet_service.cc:1514] Processing DeleteTablet for tablet 816d0f94341e409f83854640b9a8a1f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:09:54 UTC) from {username='slave'} at 127.0.0.1:58332
I20260807 15:09:54.397387 23512 tablet_replica.cc:330] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93: stopping tablet replica
I20260807 15:09:54.397580 23509 tablet_replica.cc:330] T 816d0f94341e409f83854640b9a8a1f4 P 006e74f910264e5598ec3da6508b29f5: stopping tablet replica
I20260807 15:09:54.397784 23512 raft_consensus.cc:2229] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:09:54.397917 23509 raft_consensus.cc:2229] T 816d0f94341e409f83854640b9a8a1f4 P 006e74f910264e5598ec3da6508b29f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:09:54.398228 23512 raft_consensus.cc:2258] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:54.398285 23509 raft_consensus.cc:2258] T 816d0f94341e409f83854640b9a8a1f4 P 006e74f910264e5598ec3da6508b29f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:09:54.398306 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 22858
I20260807 15:09:54.409668 23509 ts_tablet_manager.cc:1905] T 816d0f94341e409f83854640b9a8a1f4 P 006e74f910264e5598ec3da6508b29f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:54.411283 23512 ts_tablet_manager.cc:1905] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:09:54.414204 23509 ts_tablet_manager.cc:1918] T 816d0f94341e409f83854640b9a8a1f4 P 006e74f910264e5598ec3da6508b29f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:09:54.414343 23509 log.cc:1196] T 816d0f94341e409f83854640b9a8a1f4 P 006e74f910264e5598ec3da6508b29f5: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/816d0f94341e409f83854640b9a8a1f4
I20260807 15:09:54.414254 23512 ts_tablet_manager.cc:1918] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:09:54.414582 23512 log.cc:1196] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/816d0f94341e409f83854640b9a8a1f4
I20260807 15:09:54.414773 23509 ts_tablet_manager.cc:1939] T 816d0f94341e409f83854640b9a8a1f4 P 006e74f910264e5598ec3da6508b29f5: Deleting consensus metadata
I20260807 15:09:54.415091 23512 ts_tablet_manager.cc:1939] T 816d0f94341e409f83854640b9a8a1f4 P bd8fa3e16bdd49e0b2bc6d0c20387d93: Deleting consensus metadata
W20260807 15:09:54.416249 22760 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.226.65:44839 (error 108)
I20260807 15:09:54.416320 22759 catalog_manager.cc:4841] TS 006e74f910264e5598ec3da6508b29f5 (127.15.226.66:45937): tablet 816d0f94341e409f83854640b9a8a1f4 (table test-table [id=dc5c0148d90f4d82acedefbf48573438]) successfully deleted
I20260807 15:09:54.416750 22757 catalog_manager.cc:4841] TS bd8fa3e16bdd49e0b2bc6d0c20387d93 (127.15.226.67:42497): tablet 816d0f94341e409f83854640b9a8a1f4 (table test-table [id=dc5c0148d90f4d82acedefbf48573438]) successfully deleted
I20260807 15:09:54.417526 23452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:54.417182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:44839 (local address 127.0.0.1:48282)
0807 15:09:54.417323 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:54.417407 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.65:44839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 15:09:54.417675 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 23054
W20260807 15:09:54.417752 22760 catalog_manager.cc:4584] TS f6a250676d964318914683e10b1c80d8 (127.15.226.65:44839): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 816d0f94341e409f83854640b9a8a1f4: Network error: Client connection negotiation failed: client connection to 127.15.226.65:44839: connect: Connection refused (error 111)
I20260807 15:09:54.447342 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 23246
I20260807 15:09:54.460510 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 22738
2026-08-07T15:09:54Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1207 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 15:09:54.489315 16265 mini_oidc.cc:152] Starting JWKS server
I20260807 15:09:54.491432 16265 webserver.cc:460] Webserver started at https://127.0.0.1:39279/ using document root <none> and password file <none>
I20260807 15:09:54.491631 16265 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 15:09:54.491986 16265 webserver.cc:460] Webserver started at http://127.0.0.1:38913/ using document root <none> and password file <none>
2026-08-07T15:09:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:09:54Z Disabled control of system clock
I20260807 15:09:54.503894 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:43881
--webserver_interface=127.15.226.126
--webserver_port=0
--builtin_ntp_servers=127.15.226.84:41769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:43881
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39279/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115365131938-16265-0/testchainca.cert with env {}
W20260807 15:09:54.625253 23522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:54.625577 23522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:54.625622 23522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:54.629773 23522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:54.629873 23522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:54.629890 23522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:54.629908 23522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:54.629933 23522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 15:09:54.629951 23522 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 15:09:54.629966 23522 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39279/jwks.json
I20260807 15:09:54.635797 23522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:41769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:43881
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:43881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39279/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115365131938-16265-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:54.636970 23522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:54.638036 23522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:54.647603 23531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:54.647558 23535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:54.647627 23530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:54.647789 23522 server_base.cc:1034] running on GCE node
I20260807 15:09:54.648661 23522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:54.649878 23522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:54.651125 23522 hybrid_clock.cc:648] HybridClock initialized: now 1786115394651088 us; error 57 us; skew 500 ppm
I20260807 15:09:54.653366 23522 webserver.cc:460] Webserver started at http://127.15.226.126:33503/ using document root <none> and password file <none>
I20260807 15:09:54.654067 23522 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:54.654182 23522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:54.654443 23522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:54.656826 23522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "a2f3ad73bdb448f786b1724fc2c26632"
format_stamp: "Formatted at 2026-08-07 15:09:54 on dist-test-slave-3kk6"
server_key: "a12a54c43cca1ebd23aac2634acd6c16"
server_key_iv: "e3b3c66ae53a4cdab89d8c929b84f1f7"
server_key_version: "encryptionkey@0"
I20260807 15:09:54.657482 23522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "a2f3ad73bdb448f786b1724fc2c26632"
format_stamp: "Formatted at 2026-08-07 15:09:54 on dist-test-slave-3kk6"
server_key: "a12a54c43cca1ebd23aac2634acd6c16"
server_key_iv: "e3b3c66ae53a4cdab89d8c929b84f1f7"
server_key_version: "encryptionkey@0"
I20260807 15:09:54.661609 23522 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:09:54.664292 23541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:54.665774 23522 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:09:54.665948 23522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "a2f3ad73bdb448f786b1724fc2c26632"
format_stamp: "Formatted at 2026-08-07 15:09:54 on dist-test-slave-3kk6"
server_key: "a12a54c43cca1ebd23aac2634acd6c16"
server_key_iv: "e3b3c66ae53a4cdab89d8c929b84f1f7"
server_key_version: "encryptionkey@0"
I20260807 15:09:54.666083 23522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:54.681044 23522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:54.681854 23522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:54.682081 23522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:54.689947 23522 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:43881
I20260807 15:09:54.689983 23607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:43881 every 8 connection(s)
I20260807 15:09:54.690971 23522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:09:54.691840 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 23522
I20260807 15:09:54.692021 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:09:54.692385 16265 external_mini_cluster.cc:1428] Setting key 8b007eee16e034970980e84960e7463c
I20260807 15:09:54.695173 23608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:54.702231 23608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2f3ad73bdb448f786b1724fc2c26632: Bootstrap starting.
I20260807 15:09:54.702793 23612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:54.693982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51088 (local address 127.15.226.126:43881)
0807 15:09:54.694723 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:54.694749 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:54.694797 (+ 48us) server_negotiation.cc:407] Connection header received
0807 15:09:54.695570 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:54.695588 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:54.696040 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:54.696340 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:54.696717 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:54.697694 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:54.699095 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:54.699620 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:54.699755 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:54.700737 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:54.700743 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:54.700757 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:09:54.701203 (+ 446us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:54.701261 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:09:54.701282 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:54.701365 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:54.701548 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 15:09:54.701901 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":158,"threads_started":1}
I20260807 15:09:54.705087 23608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2f3ad73bdb448f786b1724fc2c26632: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:54.706014 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:09:54.706111 23608 log.cc:824] T 00000000000000000000000000000000 P a2f3ad73bdb448f786b1724fc2c26632: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:54.708595 23608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2f3ad73bdb448f786b1724fc2c26632: No bootstrap required, opened a new log
I20260807 15:09:54.712548 23608 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2f3ad73bdb448f786b1724fc2c26632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2f3ad73bdb448f786b1724fc2c26632" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 43881 } }
I20260807 15:09:54.712849 23608 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2f3ad73bdb448f786b1724fc2c26632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:54.712939 23608 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2f3ad73bdb448f786b1724fc2c26632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2f3ad73bdb448f786b1724fc2c26632, State: Initialized, Role: FOLLOWER
I20260807 15:09:54.713434 23608 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2f3ad73bdb448f786b1724fc2c26632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2f3ad73bdb448f786b1724fc2c26632" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 43881 } }
I20260807 15:09:54.713617 23608 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2f3ad73bdb448f786b1724fc2c26632 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:09:54.713697 23608 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2f3ad73bdb448f786b1724fc2c26632 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:09:54.713819 23608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2f3ad73bdb448f786b1724fc2c26632 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:54.715109 23608 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2f3ad73bdb448f786b1724fc2c26632 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2f3ad73bdb448f786b1724fc2c26632" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 43881 } }
I20260807 15:09:54.715500 23608 leader_election.cc:302] T 00000000000000000000000000000000 P a2f3ad73bdb448f786b1724fc2c26632 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2f3ad73bdb448f786b1724fc2c26632; no voters:
I20260807 15:09:54.715860 23608 leader_election.cc:288] T 00000000000000000000000000000000 P a2f3ad73bdb448f786b1724fc2c26632 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:09:54.716034 23614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2f3ad73bdb448f786b1724fc2c26632 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:54.716312 23614 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2f3ad73bdb448f786b1724fc2c26632 [term 1 LEADER]: Becoming Leader. State: Replica: a2f3ad73bdb448f786b1724fc2c26632, State: Running, Role: LEADER
I20260807 15:09:54.716671 23614 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2f3ad73bdb448f786b1724fc2c26632 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2f3ad73bdb448f786b1724fc2c26632" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 43881 } }
I20260807 15:09:54.717319 23608 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2f3ad73bdb448f786b1724fc2c26632 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:54.718688 23618 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2f3ad73bdb448f786b1724fc2c26632 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2f3ad73bdb448f786b1724fc2c26632. Latest consensus state: current_term: 1 leader_uuid: "a2f3ad73bdb448f786b1724fc2c26632" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2f3ad73bdb448f786b1724fc2c26632" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 43881 } } }
I20260807 15:09:54.718873 23618 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2f3ad73bdb448f786b1724fc2c26632 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:54.718685 23616 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2f3ad73bdb448f786b1724fc2c26632 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2f3ad73bdb448f786b1724fc2c26632" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2f3ad73bdb448f786b1724fc2c26632" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 43881 } } }
I20260807 15:09:54.719240 23616 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2f3ad73bdb448f786b1724fc2c26632 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:54.719419 23629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:09:54.722262 23629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:09:54.728132 23629 catalog_manager.cc:1347] Generated new cluster ID: 43e4ebe8cede4ae19346adf582e677db
I20260807 15:09:54.728222 23629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:09:54.745177 23629 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:09:54.746606 23629 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:09:54.759810 23629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2f3ad73bdb448f786b1724fc2c26632: Generated new TSK 0
I20260807 15:09:54.760738 23629 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 15:09:54.801086 23655 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 15:09:54.801352 23655 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 15:09:54.802109 23612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:54.781369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51094 (local address 127.15.226.126:43881)
0807 15:09:54.781534 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:54.781540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:54.781696 (+ 156us) server_negotiation.cc:407] Connection header received
0807 15:09:54.781976 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:54.781982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:54.782043 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:54.782158 (+ 115us) server_negotiation.cc:226] Negotiated authn=JWT
0807 15:09:54.782486 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:54.783704 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:54.785036 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:54.785494 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:54.785629 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:54.786097 (+ 468us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 15:09:54.801557 (+ 15460us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 15:09:54.801943 (+ 386us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51094: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":69}
W20260807 15:09:54.802198 23612 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51094: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 15:09:54.803634 23653 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:09:54.781182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:43881 (local address 127.0.0.1:51094)
0807 15:09:54.781592 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:54.781607 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:54.781785 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:54.782190 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:54.782194 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:54.782203 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0807 15:09:54.782354 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:54.782361 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:54.783857 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:54.783864 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:54.784816 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:54.784830 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:54.785639 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:54.785643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:54.785782 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:54.785840 (+ 58us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 15:09:54.801855 (+ 16015us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 15:09:54.802131 (+ 276us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.226.126:43881: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":135,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.15.226.126:43881: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39279/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 15:09:54.804638 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 23522
2026-08-07T15:09:54Z chronyd exiting
I20260807 15:09:54.852310 16265 test_util.cc:175] -----------------------------------------------
I20260807 15:09:54.852451 16265 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786115365131938-16265-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (365 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 15:09:54.853773 16265 mini_oidc.cc:152] Starting JWKS server
I20260807 15:09:54.854847 16265 webserver.cc:460] Webserver started at https://127.0.0.1:38611/ using document root <none> and password file <none>
I20260807 15:09:54.854996 16265 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 15:09:54.855310 16265 webserver.cc:460] Webserver started at http://127.0.0.1:39449/ using document root <none> and password file <none>
2026-08-07T15:09:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:09:54Z Disabled control of system clock
I20260807 15:09:54.866838 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:43521
--webserver_interface=127.15.226.126
--webserver_port=0
--builtin_ntp_servers=127.15.226.84:41327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:43521
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38611/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115365131938-16265-0/testchainca.cert with env {}
W20260807 15:09:54.988801 23663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:54.989195 23663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:54.989301 23663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:54.993719 23663 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:54.993855 23663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:54.993891 23663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:54.993930 23663 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:54.993968 23663 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 15:09:54.994000 23663 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 15:09:54.994031 23663 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38611/jwks.json
I20260807 15:09:55.000097 23663 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:41327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:43521
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:43521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38611/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115365131938-16265-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:55.001430 23663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:55.002681 23663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:55.012360 23668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:55.012488 23663 server_base.cc:1034] running on GCE node
W20260807 15:09:55.012360 23669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:55.012392 23671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:55.013332 23663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:55.014704 23663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:55.015933 23663 hybrid_clock.cc:648] HybridClock initialized: now 1786115395015922 us; error 35 us; skew 500 ppm
I20260807 15:09:55.018167 23663 webserver.cc:460] Webserver started at http://127.15.226.126:44037/ using document root <none> and password file <none>
I20260807 15:09:55.018880 23663 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:55.018963 23663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:55.019151 23663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:55.021452 23663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "084d79c2ceb1428aaf335be60ee4134b"
format_stamp: "Formatted at 2026-08-07 15:09:55 on dist-test-slave-3kk6"
server_key: "4e0b83988ad5caa5b1ced06375ecca09"
server_key_iv: "35462e53ec24dd2034c33630aae68069"
server_key_version: "encryptionkey@0"
I20260807 15:09:55.022053 23663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "084d79c2ceb1428aaf335be60ee4134b"
format_stamp: "Formatted at 2026-08-07 15:09:55 on dist-test-slave-3kk6"
server_key: "4e0b83988ad5caa5b1ced06375ecca09"
server_key_iv: "35462e53ec24dd2034c33630aae68069"
server_key_version: "encryptionkey@0"
I20260807 15:09:55.026259 23663 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260807 15:09:55.029165 23682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:55.030974 23663 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:09:55.031121 23663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "084d79c2ceb1428aaf335be60ee4134b"
format_stamp: "Formatted at 2026-08-07 15:09:55 on dist-test-slave-3kk6"
server_key: "4e0b83988ad5caa5b1ced06375ecca09"
server_key_iv: "35462e53ec24dd2034c33630aae68069"
server_key_version: "encryptionkey@0"
I20260807 15:09:55.031248 23663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:55.053614 23663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:55.054555 23663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:55.054834 23663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:55.063346 23663 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:43521
I20260807 15:09:55.063356 23761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:43521 every 8 connection(s)
I20260807 15:09:55.064602 23663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:09:55.068248 23763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:55.074390 23763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 084d79c2ceb1428aaf335be60ee4134b: Bootstrap starting.
I20260807 15:09:55.074795 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 23663
I20260807 15:09:55.074946 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:09:55.075275 16265 external_mini_cluster.cc:1428] Setting key 6421a9b2a0ffe08f9be4fa495fc6e023
I20260807 15:09:55.077450 23763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 084d79c2ceb1428aaf335be60ee4134b: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:55.078431 23763 log.cc:824] T 00000000000000000000000000000000 P 084d79c2ceb1428aaf335be60ee4134b: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:55.081033 23763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 084d79c2ceb1428aaf335be60ee4134b: No bootstrap required, opened a new log
I20260807 15:09:55.085865 23763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 084d79c2ceb1428aaf335be60ee4134b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "084d79c2ceb1428aaf335be60ee4134b" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 43521 } }
I20260807 15:09:55.086234 23763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 084d79c2ceb1428aaf335be60ee4134b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:55.086303 23763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 084d79c2ceb1428aaf335be60ee4134b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 084d79c2ceb1428aaf335be60ee4134b, State: Initialized, Role: FOLLOWER
I20260807 15:09:55.086850 23763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 084d79c2ceb1428aaf335be60ee4134b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "084d79c2ceb1428aaf335be60ee4134b" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 43521 } }
I20260807 15:09:55.087049 23763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 084d79c2ceb1428aaf335be60ee4134b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:09:55.087112 23763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 084d79c2ceb1428aaf335be60ee4134b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:09:55.087232 23763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 084d79c2ceb1428aaf335be60ee4134b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:55.087719 23767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:55.076826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60678 (local address 127.15.226.126:43521)
0807 15:09:55.077732 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:55.077755 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:55.077808 (+ 53us) server_negotiation.cc:407] Connection header received
0807 15:09:55.078815 (+ 1007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:55.078842 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:55.079218 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:55.079598 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:55.080065 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:55.081539 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:55.082891 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:55.083632 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:55.083831 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:55.085047 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:55.085054 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:55.085075 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:09:55.085644 (+ 569us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:55.085739 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:09:55.085769 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:55.085881 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:55.086133 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 15:09:55.086462 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":189,"threads_started":1}
I20260807 15:09:55.088517 23763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 084d79c2ceb1428aaf335be60ee4134b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "084d79c2ceb1428aaf335be60ee4134b" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 43521 } }
I20260807 15:09:55.088977 23763 leader_election.cc:302] T 00000000000000000000000000000000 P 084d79c2ceb1428aaf335be60ee4134b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 084d79c2ceb1428aaf335be60ee4134b; no voters:
I20260807 15:09:55.089406 23763 leader_election.cc:288] T 00000000000000000000000000000000 P 084d79c2ceb1428aaf335be60ee4134b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:09:55.089551 23769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 084d79c2ceb1428aaf335be60ee4134b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:55.089821 23769 raft_consensus.cc:686] T 00000000000000000000000000000000 P 084d79c2ceb1428aaf335be60ee4134b [term 1 LEADER]: Becoming Leader. State: Replica: 084d79c2ceb1428aaf335be60ee4134b, State: Running, Role: LEADER
I20260807 15:09:55.090202 23769 consensus_queue.cc:237] T 00000000000000000000000000000000 P 084d79c2ceb1428aaf335be60ee4134b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "084d79c2ceb1428aaf335be60ee4134b" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 43521 } }
I20260807 15:09:55.090871 23763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 084d79c2ceb1428aaf335be60ee4134b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:55.091310 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:09:55.092998 23774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 084d79c2ceb1428aaf335be60ee4134b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 084d79c2ceb1428aaf335be60ee4134b. Latest consensus state: current_term: 1 leader_uuid: "084d79c2ceb1428aaf335be60ee4134b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "084d79c2ceb1428aaf335be60ee4134b" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 43521 } } }
I20260807 15:09:55.093137 23774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 084d79c2ceb1428aaf335be60ee4134b [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:55.093428 23772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 084d79c2ceb1428aaf335be60ee4134b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "084d79c2ceb1428aaf335be60ee4134b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "084d79c2ceb1428aaf335be60ee4134b" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 43521 } } }
I20260807 15:09:55.093536 23772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 084d79c2ceb1428aaf335be60ee4134b [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:55.093719 23797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:09:55.096906 23797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:09:55.103229 23797 catalog_manager.cc:1347] Generated new cluster ID: b28ce29dea494c5ab53292782eff74ba
I20260807 15:09:55.103333 23797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:09:55.118250 23797 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:09:55.119343 23797 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:09:55.127033 23797 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 084d79c2ceb1428aaf335be60ee4134b: Generated new TSK 0
I20260807 15:09:55.127997 23797 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 15:09:55.161648 23809 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 15:09:55.161821 23809 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 15:09:55.162652 23767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:55.144065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60682 (local address 127.15.226.126:43521)
0807 15:09:55.144226 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:55.144231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:55.144419 (+ 188us) server_negotiation.cc:407] Connection header received
0807 15:09:55.144609 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:55.144613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:55.144676 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:55.144817 (+ 141us) server_negotiation.cc:226] Negotiated authn=JWT
0807 15:09:55.145036 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:55.145909 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:55.146949 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:55.147313 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:55.147424 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:55.147803 (+ 379us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 15:09:55.162139 (+ 14336us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38611/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 15:09:55.162472 (+ 333us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60682: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38611/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":63}
W20260807 15:09:55.162778 23767 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60682: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38611/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 15:09:55.162828 23808 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:09:55.143955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:43521 (local address 127.0.0.1:60682)
0807 15:09:55.144318 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:55.144333 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:55.144464 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:55.144813 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:55.144815 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:55.144821 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0807 15:09:55.144925 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:55.144930 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:55.146048 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:55.146050 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:55.146819 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:55.146832 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:55.147483 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:55.147486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:55.147561 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:55.147589 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 15:09:55.162372 (+ 14783us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38611/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 15:09:55.162591 (+ 219us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.226.126:43521: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38611/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":121,"threads_started":1}
I20260807 15:09:55.163627 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 23663
2026-08-07T15:09:55Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (357 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 15:09:55.210971 16265 mini_oidc.cc:152] Starting JWKS server
I20260807 15:09:55.212001 16265 webserver.cc:460] Webserver started at https://127.0.0.1:35213/ using document root <none> and password file <none>
I20260807 15:09:55.212095 16265 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 15:09:55.212318 16265 webserver.cc:460] Webserver started at http://127.0.0.1:45731/ using document root <none> and password file <none>
2026-08-07T15:09:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:09:55Z Disabled control of system clock
I20260807 15:09:55.224319 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:34851
--webserver_interface=127.15.226.126
--webserver_port=0
--builtin_ntp_servers=127.15.226.84:42567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:34851
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35213/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 15:09:55.343873 23817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:55.344231 23817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:55.344332 23817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:55.348587 23817 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:55.348701 23817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:55.348733 23817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:55.348775 23817 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:55.348812 23817 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 15:09:55.348850 23817 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 15:09:55.348877 23817 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35213/jwks.json
I20260807 15:09:55.353948 23817 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:42567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:34851
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:34851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35213/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:55.355281 23817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:55.356792 23817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:55.366117 23825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:55.366066 23827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:55.366082 23829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:55.366235 23817 server_base.cc:1034] running on GCE node
I20260807 15:09:55.367233 23817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:55.368553 23817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:55.369818 23817 hybrid_clock.cc:648] HybridClock initialized: now 1786115395369795 us; error 43 us; skew 500 ppm
I20260807 15:09:55.372066 23817 webserver.cc:460] Webserver started at http://127.15.226.126:34967/ using document root <none> and password file <none>
I20260807 15:09:55.372704 23817 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:55.372874 23817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:55.373140 23817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:55.375427 23817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "f8df21522b514a07b29108e3c441f2b6"
format_stamp: "Formatted at 2026-08-07 15:09:55 on dist-test-slave-3kk6"
server_key: "ead3adcd9b3ff49ec423c97f75b2b74d"
server_key_iv: "b17baac8a9425403fbb7f1f720c0da21"
server_key_version: "encryptionkey@0"
I20260807 15:09:55.376057 23817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "f8df21522b514a07b29108e3c441f2b6"
format_stamp: "Formatted at 2026-08-07 15:09:55 on dist-test-slave-3kk6"
server_key: "ead3adcd9b3ff49ec423c97f75b2b74d"
server_key_iv: "b17baac8a9425403fbb7f1f720c0da21"
server_key_version: "encryptionkey@0"
I20260807 15:09:55.380323 23817 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:09:55.383114 23839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:55.384456 23817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:09:55.384613 23817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "f8df21522b514a07b29108e3c441f2b6"
format_stamp: "Formatted at 2026-08-07 15:09:55 on dist-test-slave-3kk6"
server_key: "ead3adcd9b3ff49ec423c97f75b2b74d"
server_key_iv: "b17baac8a9425403fbb7f1f720c0da21"
server_key_version: "encryptionkey@0"
I20260807 15:09:55.384756 23817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:55.414062 23817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:55.414985 23817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:55.415243 23817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:55.423101 23817 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:34851
I20260807 15:09:55.423128 23913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:34851 every 8 connection(s)
I20260807 15:09:55.424158 23817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:09:55.427762 23914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:55.432056 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 23817
I20260807 15:09:55.432205 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:09:55.432526 16265 external_mini_cluster.cc:1428] Setting key c0f987e7b115deb4ee09e3555f989d67
I20260807 15:09:55.434391 23914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8df21522b514a07b29108e3c441f2b6: Bootstrap starting.
I20260807 15:09:55.436761 23914 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8df21522b514a07b29108e3c441f2b6: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:55.437546 23914 log.cc:824] T 00000000000000000000000000000000 P f8df21522b514a07b29108e3c441f2b6: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:55.439585 23914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8df21522b514a07b29108e3c441f2b6: No bootstrap required, opened a new log
I20260807 15:09:55.442346 23917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:55.434002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44296 (local address 127.15.226.126:34851)
0807 15:09:55.434871 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:55.434891 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:55.434931 (+ 40us) server_negotiation.cc:407] Connection header received
0807 15:09:55.435605 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:55.435621 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:55.436025 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:55.436385 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:55.436861 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:55.437945 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:55.438943 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:55.439468 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:55.439653 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:55.440597 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:55.440602 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:55.440615 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:09:55.441018 (+ 403us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:55.441073 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:09:55.441093 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:55.441208 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:55.441338 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 15:09:55.441603 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":172,"threads_started":1}
I20260807 15:09:55.443431 23914 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8df21522b514a07b29108e3c441f2b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8df21522b514a07b29108e3c441f2b6" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34851 } }
I20260807 15:09:55.443694 23914 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8df21522b514a07b29108e3c441f2b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:55.443782 23914 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8df21522b514a07b29108e3c441f2b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8df21522b514a07b29108e3c441f2b6, State: Initialized, Role: FOLLOWER
I20260807 15:09:55.444274 23914 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8df21522b514a07b29108e3c441f2b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8df21522b514a07b29108e3c441f2b6" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34851 } }
I20260807 15:09:55.444468 23914 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8df21522b514a07b29108e3c441f2b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:09:55.444545 23914 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8df21522b514a07b29108e3c441f2b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:09:55.444674 23914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8df21522b514a07b29108e3c441f2b6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:55.445504 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:09:55.445923 23914 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8df21522b514a07b29108e3c441f2b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8df21522b514a07b29108e3c441f2b6" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34851 } }
I20260807 15:09:55.446278 23914 leader_election.cc:302] T 00000000000000000000000000000000 P f8df21522b514a07b29108e3c441f2b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8df21522b514a07b29108e3c441f2b6; no voters:
I20260807 15:09:55.446691 23914 leader_election.cc:288] T 00000000000000000000000000000000 P f8df21522b514a07b29108e3c441f2b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:09:55.446817 23919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8df21522b514a07b29108e3c441f2b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:55.447059 23919 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8df21522b514a07b29108e3c441f2b6 [term 1 LEADER]: Becoming Leader. State: Replica: f8df21522b514a07b29108e3c441f2b6, State: Running, Role: LEADER
I20260807 15:09:55.447408 23919 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8df21522b514a07b29108e3c441f2b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8df21522b514a07b29108e3c441f2b6" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34851 } }
I20260807 15:09:55.448055 23914 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8df21522b514a07b29108e3c441f2b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:55.448514 23919 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8df21522b514a07b29108e3c441f2b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8df21522b514a07b29108e3c441f2b6. Latest consensus state: current_term: 1 leader_uuid: "f8df21522b514a07b29108e3c441f2b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8df21522b514a07b29108e3c441f2b6" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34851 } } }
I20260807 15:09:55.448649 23919 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8df21522b514a07b29108e3c441f2b6 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:55.448963 23920 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8df21522b514a07b29108e3c441f2b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8df21522b514a07b29108e3c441f2b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8df21522b514a07b29108e3c441f2b6" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34851 } } }
I20260807 15:09:55.449079 23923 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:09:55.449115 23920 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8df21522b514a07b29108e3c441f2b6 [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:55.452490 23923 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:09:55.458673 23923 catalog_manager.cc:1347] Generated new cluster ID: 863168b3e8ff41969837e7c5f220b7aa
I20260807 15:09:55.458842 23923 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:09:55.466400 23923 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:09:55.467756 23923 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:09:55.478683 23923 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8df21522b514a07b29108e3c441f2b6: Generated new TSK 0
I20260807 15:09:55.479552 23923 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:09:55.497116 23917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:55.491179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44298 (local address 127.15.226.126:34851)
0807 15:09:55.491342 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:55.491350 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:55.491521 (+ 171us) server_negotiation.cc:407] Connection header received
0807 15:09:55.491668 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:55.491672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:55.491750 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:55.491886 (+ 136us) server_negotiation.cc:226] Negotiated authn=JWT
0807 15:09:55.492186 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:55.493054 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:55.494068 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:55.494458 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:55.494565 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:55.495021 (+ 456us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 15:09:55.496681 (+ 1660us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35213/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 15:09:55.496949 (+ 268us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44298: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35213/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 15:09:55.497211 23917 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44298: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35213/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 15:09:55.498843 23953 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:09:55.491054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34851 (local address 127.0.0.1:44298)
0807 15:09:55.491423 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:55.491438 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:55.491547 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:55.491933 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:55.491935 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:55.491943 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0807 15:09:55.492056 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:55.492062 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:55.493199 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:55.493203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:55.493929 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:55.493941 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:55.494598 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:55.494601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:55.494733 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:55.494771 (+ 38us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 15:09:55.496880 (+ 2109us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35213/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 15:09:55.498630 (+ 1750us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.226.126:34851: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35213/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":1518,"thread_start_us":117,"threads_started":1}
I20260807 15:09:55.499769 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 23817
E20260807 15:09:55.509776 23954 webserver.cc:533] Webserver: sslize failed:
E20260807 15:09:55.509917 23954 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-07T15:09:55Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (333 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T15:09:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:09:55Z Disabled control of system clock
I20260807 15:09:55.555943 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:36435
--webserver_interface=127.15.226.126
--webserver_port=0
--builtin_ntp_servers=127.15.226.84:46251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:36435
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115365131938-16265-0/testchainca.cert with env {}
W20260807 15:09:55.676599 23961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:55.676893 23961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:55.676936 23961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:55.681084 23961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:55.681164 23961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:55.681180 23961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:55.681196 23961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:55.681212 23961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 15:09:55.681228 23961 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 15:09:55.681242 23961 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 15:09:55.686833 23961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:46251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:36435
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:36435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115365131938-16265-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:55.687902 23961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:55.688920 23961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:55.698464 23972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:55.698467 23970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:55.698542 23961 server_base.cc:1034] running on GCE node
W20260807 15:09:55.698464 23976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:55.699378 23961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:55.700587 23961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:55.701778 23961 hybrid_clock.cc:648] HybridClock initialized: now 1786115395701761 us; error 34 us; skew 500 ppm
I20260807 15:09:55.703874 23961 webserver.cc:460] Webserver started at http://127.15.226.126:36557/ using document root <none> and password file <none>
I20260807 15:09:55.704505 23961 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:55.704609 23961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:55.704854 23961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:55.707193 23961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "593d1cf04530429baa80c14449a999dc"
format_stamp: "Formatted at 2026-08-07 15:09:55 on dist-test-slave-3kk6"
server_key: "64dc28ba29abab1d6bbedec1dbca0db9"
server_key_iv: "9ebc2c5018cff2fca090bd5fb5e428d8"
server_key_version: "encryptionkey@0"
I20260807 15:09:55.707854 23961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "593d1cf04530429baa80c14449a999dc"
format_stamp: "Formatted at 2026-08-07 15:09:55 on dist-test-slave-3kk6"
server_key: "64dc28ba29abab1d6bbedec1dbca0db9"
server_key_iv: "9ebc2c5018cff2fca090bd5fb5e428d8"
server_key_version: "encryptionkey@0"
I20260807 15:09:55.711786 23961 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:09:55.714537 23983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:55.715898 23961 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:09:55.716055 23961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "593d1cf04530429baa80c14449a999dc"
format_stamp: "Formatted at 2026-08-07 15:09:55 on dist-test-slave-3kk6"
server_key: "64dc28ba29abab1d6bbedec1dbca0db9"
server_key_iv: "9ebc2c5018cff2fca090bd5fb5e428d8"
server_key_version: "encryptionkey@0"
I20260807 15:09:55.716185 23961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:55.741881 23961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:55.742789 23961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:55.743021 23961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:55.751094 23961 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:36435
I20260807 15:09:55.751124 24052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:36435 every 8 connection(s)
I20260807 15:09:55.752221 23961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:09:55.753674 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 23961
I20260807 15:09:55.753832 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:09:55.754176 16265 external_mini_cluster.cc:1428] Setting key 4ef60290038181374194f4ebf1e02793
I20260807 15:09:55.756212 24053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:55.762633 24053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 593d1cf04530429baa80c14449a999dc: Bootstrap starting.
I20260807 15:09:55.764102 24056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:55.755727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51360 (local address 127.15.226.126:36435)
0807 15:09:55.756489 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:55.756506 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:55.756545 (+ 39us) server_negotiation.cc:407] Connection header received
0807 15:09:55.757217 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:55.757233 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:55.757635 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:55.758031 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:55.758481 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:55.759552 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:55.760489 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:55.760975 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:55.761172 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:55.762202 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:55.762207 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:55.762220 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:09:55.762626 (+ 406us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:55.762747 (+ 121us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:09:55.762771 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:55.762954 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:55.763101 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 15:09:55.763379 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":156,"threads_started":1}
I20260807 15:09:55.765436 24053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 593d1cf04530429baa80c14449a999dc: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:55.766417 24053 log.cc:824] T 00000000000000000000000000000000 P 593d1cf04530429baa80c14449a999dc: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:55.767390 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:09:55.769042 24053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 593d1cf04530429baa80c14449a999dc: No bootstrap required, opened a new log
I20260807 15:09:55.773102 24053 raft_consensus.cc:348] T 00000000000000000000000000000000 P 593d1cf04530429baa80c14449a999dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "593d1cf04530429baa80c14449a999dc" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36435 } }
I20260807 15:09:55.773427 24053 raft_consensus.cc:374] T 00000000000000000000000000000000 P 593d1cf04530429baa80c14449a999dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:55.773496 24053 raft_consensus.cc:729] T 00000000000000000000000000000000 P 593d1cf04530429baa80c14449a999dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 593d1cf04530429baa80c14449a999dc, State: Initialized, Role: FOLLOWER
I20260807 15:09:55.773903 24053 consensus_queue.cc:260] T 00000000000000000000000000000000 P 593d1cf04530429baa80c14449a999dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "593d1cf04530429baa80c14449a999dc" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36435 } }
I20260807 15:09:55.774032 24053 raft_consensus.cc:388] T 00000000000000000000000000000000 P 593d1cf04530429baa80c14449a999dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:09:55.774082 24053 raft_consensus.cc:482] T 00000000000000000000000000000000 P 593d1cf04530429baa80c14449a999dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:09:55.774163 24053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 593d1cf04530429baa80c14449a999dc [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:55.775617 24053 raft_consensus.cc:504] T 00000000000000000000000000000000 P 593d1cf04530429baa80c14449a999dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "593d1cf04530429baa80c14449a999dc" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36435 } }
I20260807 15:09:55.775949 24053 leader_election.cc:302] T 00000000000000000000000000000000 P 593d1cf04530429baa80c14449a999dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 593d1cf04530429baa80c14449a999dc; no voters:
I20260807 15:09:55.776283 24053 leader_election.cc:288] T 00000000000000000000000000000000 P 593d1cf04530429baa80c14449a999dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:09:55.776362 24059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 593d1cf04530429baa80c14449a999dc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:55.776690 24059 raft_consensus.cc:686] T 00000000000000000000000000000000 P 593d1cf04530429baa80c14449a999dc [term 1 LEADER]: Becoming Leader. State: Replica: 593d1cf04530429baa80c14449a999dc, State: Running, Role: LEADER
I20260807 15:09:55.777024 24059 consensus_queue.cc:237] T 00000000000000000000000000000000 P 593d1cf04530429baa80c14449a999dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "593d1cf04530429baa80c14449a999dc" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36435 } }
I20260807 15:09:55.777732 24053 sys_catalog.cc:564] T 00000000000000000000000000000000 P 593d1cf04530429baa80c14449a999dc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:55.778972 24061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 593d1cf04530429baa80c14449a999dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 593d1cf04530429baa80c14449a999dc. Latest consensus state: current_term: 1 leader_uuid: "593d1cf04530429baa80c14449a999dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "593d1cf04530429baa80c14449a999dc" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36435 } } }
I20260807 15:09:55.779019 24060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 593d1cf04530429baa80c14449a999dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "593d1cf04530429baa80c14449a999dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "593d1cf04530429baa80c14449a999dc" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36435 } } }
I20260807 15:09:55.779112 24061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 593d1cf04530429baa80c14449a999dc [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:55.779142 24060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 593d1cf04530429baa80c14449a999dc [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:55.779755 24068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:09:55.783390 24068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:09:55.789880 24068 catalog_manager.cc:1347] Generated new cluster ID: df5d5c6fbcb6467eb5d50168c5100ffa
I20260807 15:09:55.789978 24068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:09:55.805807 24068 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:09:55.807235 24068 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:09:55.816761 24068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 593d1cf04530429baa80c14449a999dc: Generated new TSK 0
I20260807 15:09:55.817724 24068 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:09:56.715538 24056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:55.832253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51376 (local address 127.15.226.126:36435)
0807 15:09:55.832422 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:55.832427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:55.832615 (+ 188us) server_negotiation.cc:407] Connection header received
0807 15:09:55.832823 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:55.832825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:55.832899 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:55.832994 (+ 95us) server_negotiation.cc:226] Negotiated authn=JWT
0807 15:09:55.833278 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:55.833821 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:55.835165 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:55.835397 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:55.835492 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:55.835926 (+ 434us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 15:09:56.715095 (+879169us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 15:09:56.715409 (+ 314us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51376: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":68}
W20260807 15:09:56.715628 24056 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51376: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 15:09:56.716053 24092 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:09:55.832145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36435 (local address 127.0.0.1:51376)
0807 15:09:55.832507 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:55.832525 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:55.832658 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:55.833023 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:55.833027 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:55.833035 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0807 15:09:55.833159 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:55.833166 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:55.833950 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:55.833954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:55.835010 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:55.835024 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:55.835544 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:55.835548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:55.835645 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:55.835679 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 15:09:56.715406 (+879727us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 15:09:56.715673 (+ 267us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.226.126:36435: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":134,"threads_started":1}
I20260807 15:09:56.716722 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 23961
2026-08-07T15:09:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1199 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (134 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (136 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:09:57 dist-test-slave-3kk6 krb5kdc[24112](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:09:57 dist-test-slave-3kk6 krb5kdc[24112](info): set up 2 sockets
Aug 07 15:09:57 dist-test-slave-3kk6 krb5kdc[24112](info): commencing operation
krb5kdc: starting...
W20260807 15:09:58.240608 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:09:58 dist-test-slave-3kk6 krb5kdc[24112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:09:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:09:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:09:58.390516 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:34387
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:39815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:34387
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:58.512955 24128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:58.513366 24128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:58.513453 24128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:58.517974 24128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:09:58.518093 24128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:58.518132 24128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:58.518177 24128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:09:58.518219 24128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:09:58.524224 24128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:39815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:34387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:34387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:58.525492 24128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:58.526621 24128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:58.536800 24144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:58.536815 24141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:58.536791 24137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:58.537168 24128 server_base.cc:1034] running on GCE node
I20260807 15:09:58.537947 24128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:58.539642 24128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:58.540911 24128 hybrid_clock.cc:648] HybridClock initialized: now 1786115398540878 us; error 61 us; skew 500 ppm
Aug 07 15:09:58 dist-test-slave-3kk6 krb5kdc[24112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115398, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:58.544584 24128 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:09:58.545856 24128 webserver.cc:460] Webserver started at http://127.15.226.126:37227/ using document root <none> and password file <none>
I20260807 15:09:58.546555 24128 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:58.546708 24128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:58.547009 24128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:58.549695 24128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "f8136fecaa5345b7a6ffd55f5150782a"
format_stamp: "Formatted at 2026-08-07 15:09:58 on dist-test-slave-3kk6"
server_key: "8c70b3d5b23f2b8523b842a9418dd7b8"
server_key_iv: "47cfb9fb94c63c375664ce8ab14fd9b1"
server_key_version: "encryptionkey@0"
I20260807 15:09:58.550442 24128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "f8136fecaa5345b7a6ffd55f5150782a"
format_stamp: "Formatted at 2026-08-07 15:09:58 on dist-test-slave-3kk6"
server_key: "8c70b3d5b23f2b8523b842a9418dd7b8"
server_key_iv: "47cfb9fb94c63c375664ce8ab14fd9b1"
server_key_version: "encryptionkey@0"
I20260807 15:09:58.555460 24128 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.003s
I20260807 15:09:58.558806 24152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:58.560400 24128 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 15:09:58.560595 24128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "f8136fecaa5345b7a6ffd55f5150782a"
format_stamp: "Formatted at 2026-08-07 15:09:58 on dist-test-slave-3kk6"
server_key: "8c70b3d5b23f2b8523b842a9418dd7b8"
server_key_iv: "47cfb9fb94c63c375664ce8ab14fd9b1"
server_key_version: "encryptionkey@0"
I20260807 15:09:58.560760 24128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:58.587692 24128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:58.591974 24128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:58.592250 24128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:58.601073 24128 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:34387
I20260807 15:09:58.601104 24224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:34387 every 8 connection(s)
I20260807 15:09:58.602254 24128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:09:58.605419 24226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:09:58.609908 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 24128
I20260807 15:09:58.610056 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:09:58.610386 16265 external_mini_cluster.cc:1428] Setting key a65a99ff981501af099268836ba7fd92
I20260807 15:09:58.612032 24226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8136fecaa5345b7a6ffd55f5150782a: Bootstrap starting.
I20260807 15:09:58.614569 24226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8136fecaa5345b7a6ffd55f5150782a: Neither blocks nor log segments found. Creating new log.
I20260807 15:09:58.615509 24226 log.cc:824] T 00000000000000000000000000000000 P f8136fecaa5345b7a6ffd55f5150782a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:09:58.617698 24226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8136fecaa5345b7a6ffd55f5150782a: No bootstrap required, opened a new log
Aug 07 15:09:58 dist-test-slave-3kk6 krb5kdc[24112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:58.622630 24226 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8136fecaa5345b7a6ffd55f5150782a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8136fecaa5345b7a6ffd55f5150782a" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34387 } }
I20260807 15:09:58.622977 24226 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8136fecaa5345b7a6ffd55f5150782a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:09:58.623045 24226 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8136fecaa5345b7a6ffd55f5150782a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8136fecaa5345b7a6ffd55f5150782a, State: Initialized, Role: FOLLOWER
I20260807 15:09:58.623612 24226 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8136fecaa5345b7a6ffd55f5150782a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8136fecaa5345b7a6ffd55f5150782a" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34387 } }
I20260807 15:09:58.623785 24226 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8136fecaa5345b7a6ffd55f5150782a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:09:58.623867 24226 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8136fecaa5345b7a6ffd55f5150782a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:09:58.623982 24226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8136fecaa5345b7a6ffd55f5150782a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:09:58.625239 24226 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8136fecaa5345b7a6ffd55f5150782a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8136fecaa5345b7a6ffd55f5150782a" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34387 } }
I20260807 15:09:58.625681 24226 leader_election.cc:302] T 00000000000000000000000000000000 P f8136fecaa5345b7a6ffd55f5150782a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8136fecaa5345b7a6ffd55f5150782a; no voters:
I20260807 15:09:58.626138 24226 leader_election.cc:288] T 00000000000000000000000000000000 P f8136fecaa5345b7a6ffd55f5150782a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:09:58.626257 24234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8136fecaa5345b7a6ffd55f5150782a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:09:58.626502 24234 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8136fecaa5345b7a6ffd55f5150782a [term 1 LEADER]: Becoming Leader. State: Replica: f8136fecaa5345b7a6ffd55f5150782a, State: Running, Role: LEADER
I20260807 15:09:58.626935 24234 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8136fecaa5345b7a6ffd55f5150782a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8136fecaa5345b7a6ffd55f5150782a" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34387 } }
I20260807 15:09:58.627147 24229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:58.612260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54574 (local address 127.15.226.126:34387)
0807 15:09:58.612767 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:58.612783 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:58.612833 (+ 50us) server_negotiation.cc:407] Connection header received
0807 15:09:58.613546 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:58.613565 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:58.613938 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:58.614336 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:58.615358 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.616432 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.617391 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.617975 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.618166 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:58.621473 (+ 3307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:58.621480 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:58.621499 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:58.621535 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:58.623635 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:58.624242 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:58.624247 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:58.624252 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:58.624380 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:58.624693 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:58.624695 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:58.624697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:58.625192 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:58.625388 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:58.625675 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 15:09:58.626009 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":161,"threads_started":1}
I20260807 15:09:58.627656 24226 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8136fecaa5345b7a6ffd55f5150782a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:09:58.628273 24234 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8136fecaa5345b7a6ffd55f5150782a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8136fecaa5345b7a6ffd55f5150782a. Latest consensus state: current_term: 1 leader_uuid: "f8136fecaa5345b7a6ffd55f5150782a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8136fecaa5345b7a6ffd55f5150782a" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34387 } } }
I20260807 15:09:58.629045 24234 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8136fecaa5345b7a6ffd55f5150782a [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:58.628926 24236 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8136fecaa5345b7a6ffd55f5150782a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8136fecaa5345b7a6ffd55f5150782a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8136fecaa5345b7a6ffd55f5150782a" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34387 } } }
I20260807 15:09:58.629362 24236 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8136fecaa5345b7a6ffd55f5150782a [sys.catalog]: This master's current role is: LEADER
I20260807 15:09:58.629693 24245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:09:58.632902 24245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:09:58.639878 24245 catalog_manager.cc:1347] Generated new cluster ID: edd120fcc4364ca6875fa262cdb4ae26
I20260807 15:09:58.639995 24245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:09:58.657666 24245 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:09:58.659148 24245 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:09:58.671152 24245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8136fecaa5345b7a6ffd55f5150782a: Generated new TSK 0
I20260807 15:09:58.672224 24245 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.226.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
I20260807 15:09:58.718483 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34387
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:39815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:58.855877 24262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:58.856215 24262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:58.856266 24262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:58.860666 24262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:58.860769 24262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:58.860836 24262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:09:58.866994 24262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:39815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:58.868315 24262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:58.869481 24262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:58.880194 24270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:58.880211 24271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:58.880216 24274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:58.880334 24262 server_base.cc:1034] running on GCE node
I20260807 15:09:58.881063 24262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:58.881786 24262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:58.882983 24262 hybrid_clock.cc:648] HybridClock initialized: now 1786115398882945 us; error 53 us; skew 500 ppm
Aug 07 15:09:58 dist-test-slave-3kk6 krb5kdc[24112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115398, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:58.886184 24262 init.cc:377] Logged in from keytab as kudu/127.15.226.65@KRBTEST.COM (short username kudu)
I20260807 15:09:58.887279 24262 webserver.cc:460] Webserver started at http://127.15.226.65:44975/ using document root <none> and password file <none>
I20260807 15:09:58.888011 24262 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:58.888087 24262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:58.888314 24262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:58.890600 24262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "3c38edfca61847f6b6dc2fee26e14ba1"
format_stamp: "Formatted at 2026-08-07 15:09:58 on dist-test-slave-3kk6"
server_key: "b6c9151f9e163ed91607e934c51701e7"
server_key_iv: "cec1dab0e8565e13f6433798bdcce97b"
server_key_version: "encryptionkey@0"
I20260807 15:09:58.891300 24262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "3c38edfca61847f6b6dc2fee26e14ba1"
format_stamp: "Formatted at 2026-08-07 15:09:58 on dist-test-slave-3kk6"
server_key: "b6c9151f9e163ed91607e934c51701e7"
server_key_iv: "cec1dab0e8565e13f6433798bdcce97b"
server_key_version: "encryptionkey@0"
I20260807 15:09:58.895443 24262 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:09:58.898303 24287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:58.899883 24262 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 15:09:58.900054 24262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "3c38edfca61847f6b6dc2fee26e14ba1"
format_stamp: "Formatted at 2026-08-07 15:09:58 on dist-test-slave-3kk6"
server_key: "b6c9151f9e163ed91607e934c51701e7"
server_key_iv: "cec1dab0e8565e13f6433798bdcce97b"
server_key_version: "encryptionkey@0"
I20260807 15:09:58.900202 24262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:58.914034 24262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:58.917929 24262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:58.918190 24262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:58.918902 24262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:58.919822 24262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:58.919907 24262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:58.919981 24262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:58.920034 24262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:58.930522 24262 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:38921
I20260807 15:09:58.930543 24450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:38921 every 8 connection(s)
I20260807 15:09:58.931593 24262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:09:58.938591 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 24262
I20260807 15:09:58.938805 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:09:58.939213 16265 external_mini_cluster.cc:1428] Setting key 9ce33f35b43c14f33c2dc31eef3d2bcd
Aug 07 15:09:58 dist-test-slave-3kk6 krb5kdc[24112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115398, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:58.948263 24229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:58.933786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:58221 (local address 127.15.226.126:34387)
0807 15:09:58.933973 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:58.933977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:58.934963 (+ 986us) server_negotiation.cc:407] Connection header received
0807 15:09:58.936458 (+ 1495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:58.936461 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:58.936516 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:58.936608 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:58.938044 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.938617 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.940250 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.940601 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.940721 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:58.943985 (+ 3264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:58.943990 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:58.943993 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:58.944028 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:58.945610 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:58.946224 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:58.946228 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:58.946230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:58.946328 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:58.946892 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:58.946895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:58.946896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:58.947101 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:58.947185 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:58.948002 (+ 817us) server_negotiation.cc:299] Negotiation successful
0807 15:09:58.948129 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 15:09:58.949604 24456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:58.934196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34387 (local address 127.15.226.65:58221)
0807 15:09:58.934782 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:58.934837 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:58.936179 (+ 1342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:58.936875 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:58.936883 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:58.937308 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:58.937863 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:58.937878 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.938865 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.938871 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:58.940122 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:58.940133 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:58.940790 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:58.940794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:58.941026 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:58.941733 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:58.941761 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:58.943733 (+ 1972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:58.945760 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:58.945767 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:58.945786 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:58.946104 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:58.946471 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:58.946478 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:58.946482 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:58.946774 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:58.947225 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:58.947236 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:58.947674 (+ 438us) client_negotiation.cc:769] Sending connection context
0807 15:09:58.947931 (+ 257us) client_negotiation.cc:240] Negotiation successful
0807 15:09:58.948389 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":145,"threads_started":1}
WARNING: no policy specified for kudu/127.15.226.66@KRBTEST.COM; defaulting to no policy
I20260807 15:09:58.951125 24452 heartbeater.cc:347] Connected to a master server at 127.15.226.126:34387
I20260807 15:09:58.951404 24452 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:58.952075 24452 heartbeater.cc:511] Master 127.15.226.126:34387 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.66@KRBTEST.COM" created.
I20260807 15:09:58.953922 24178 ts_manager.cc:194] Registered new tserver with Master: 3c38edfca61847f6b6dc2fee26e14ba1 (127.15.226.65:38921)
I20260807 15:09:58.955302 24178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.65@KRBTEST.COM'} at 127.15.226.65:58221
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
I20260807 15:09:58.973603 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34387
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:39815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:59.099311 24462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:59.099692 24462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:59.099777 24462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:59.104174 24462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:59.104298 24462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:59.104404 24462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:09:59.110284 24462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:39815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:59.111582 24462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:59.112752 24462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:59.123601 24468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:59.123603 24471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:59.123878 24462 server_base.cc:1034] running on GCE node
W20260807 15:09:59.123601 24469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:59.124429 24462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:59.125180 24462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:59.126421 24462 hybrid_clock.cc:648] HybridClock initialized: now 1786115399126379 us; error 50 us; skew 500 ppm
Aug 07 15:09:59 dist-test-slave-3kk6 krb5kdc[24112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115399, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:59.129944 24462 init.cc:377] Logged in from keytab as kudu/127.15.226.66@KRBTEST.COM (short username kudu)
I20260807 15:09:59.131178 24462 webserver.cc:460] Webserver started at http://127.15.226.66:46261/ using document root <none> and password file <none>
I20260807 15:09:59.131842 24462 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:59.131917 24462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:59.132102 24462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:59.134394 24462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "da3234e1846c4c7e99a081883f24647a"
format_stamp: "Formatted at 2026-08-07 15:09:59 on dist-test-slave-3kk6"
server_key: "360ccfcb2debc8df50661bbf3f5aeaa5"
server_key_iv: "8a7afe9182ecf65a14302852b9922f3a"
server_key_version: "encryptionkey@0"
I20260807 15:09:59.135051 24462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "da3234e1846c4c7e99a081883f24647a"
format_stamp: "Formatted at 2026-08-07 15:09:59 on dist-test-slave-3kk6"
server_key: "360ccfcb2debc8df50661bbf3f5aeaa5"
server_key_iv: "8a7afe9182ecf65a14302852b9922f3a"
server_key_version: "encryptionkey@0"
I20260807 15:09:59.139120 24462 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:09:59.142035 24479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:59.143525 24462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:09:59.143652 24462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "da3234e1846c4c7e99a081883f24647a"
format_stamp: "Formatted at 2026-08-07 15:09:59 on dist-test-slave-3kk6"
server_key: "360ccfcb2debc8df50661bbf3f5aeaa5"
server_key_iv: "8a7afe9182ecf65a14302852b9922f3a"
server_key_version: "encryptionkey@0"
I20260807 15:09:59.143767 24462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:59.167982 24462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:59.171706 24462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:59.171897 24462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:59.172562 24462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:59.173419 24462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:59.173471 24462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:59.173512 24462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:59.173528 24462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:59.183219 24462 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:46557
I20260807 15:09:59.183301 24626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:46557 every 8 connection(s)
I20260807 15:09:59.184266 24462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:09:59.192685 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 24462
I20260807 15:09:59.192845 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:09:59.193146 16265 external_mini_cluster.cc:1428] Setting key 1c26e5e107c1e2f57a4c31951570c08f
Aug 07 15:09:59 dist-test-slave-3kk6 krb5kdc[24112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115399, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:59.200466 24229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:59.186284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:48739 (local address 127.15.226.126:34387)
0807 15:09:59.186455 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:59.186458 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:59.187465 (+ 1007us) server_negotiation.cc:407] Connection header received
0807 15:09:59.189064 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:59.189068 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:59.189136 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:59.189252 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:59.190562 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.191174 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.192391 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.192624 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.192762 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:59.195779 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:59.195782 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:59.195784 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:59.195810 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:59.197721 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:59.198374 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:59.198379 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:59.198383 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:59.198497 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:59.198961 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:59.198968 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:59.198976 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:59.199276 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:59.199406 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:59.199988 (+ 582us) server_negotiation.cc:299] Negotiation successful
0807 15:09:59.200182 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 15:09:59.201159 24629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:59.186729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34387 (local address 127.15.226.66:48739)
0807 15:09:59.187283 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:59.187332 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:59.188801 (+ 1469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:59.189450 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:59.189458 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:59.189874 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:59.190400 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:59.190414 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.191323 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.191326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:59.192252 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:59.192262 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.192815 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.192820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:59.193034 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:59.193803 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:59.193826 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:59.195583 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:59.197896 (+ 2313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:59.197903 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:59.197919 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:59.198252 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:59.198603 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:59.198605 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:59.198607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:59.198836 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:59.199400 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:59.199406 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:59.199693 (+ 287us) client_negotiation.cc:769] Sending connection context
0807 15:09:59.199878 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 15:09:59.200200 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":134,"threads_started":1}
I20260807 15:09:59.202492 24627 heartbeater.cc:347] Connected to a master server at 127.15.226.126:34387
I20260807 15:09:59.202795 24627 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:59.203379 24627 heartbeater.cc:511] Master 127.15.226.126:34387 requested a full tablet report, sending...
I20260807 15:09:59.204668 24178 ts_manager.cc:194] Registered new tserver with Master: da3234e1846c4c7e99a081883f24647a (127.15.226.66:46557)
WARNING: no policy specified for kudu/127.15.226.67@KRBTEST.COM; defaulting to no policy
I20260807 15:09:59.205588 24178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.66@KRBTEST.COM'} at 127.15.226.66:48739
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
I20260807 15:09:59.228245 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34387
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:39815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:09:59.359122 24633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:09:59.359467 24633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:09:59.359514 24633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:09:59.363721 24633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:09:59.363814 24633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:09:59.363878 24633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:09:59.369750 24633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:39815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:09:59.371004 24633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:09:59.372459 24633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:09:59.383953 24639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:09:59.383955 24640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:59.384297 24633 server_base.cc:1034] running on GCE node
W20260807 15:09:59.383955 24642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:09:59.384990 24633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:09:59.385774 24633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:09:59.387006 24633 hybrid_clock.cc:648] HybridClock initialized: now 1786115399386986 us; error 36 us; skew 500 ppm
Aug 07 15:09:59 dist-test-slave-3kk6 krb5kdc[24112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115399, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:59.393812 24633 init.cc:377] Logged in from keytab as kudu/127.15.226.67@KRBTEST.COM (short username kudu)
I20260807 15:09:59.395079 24633 webserver.cc:460] Webserver started at http://127.15.226.67:39555/ using document root <none> and password file <none>
I20260807 15:09:59.395790 24633 fs_manager.cc:359] Metadata directory not provided
I20260807 15:09:59.395922 24633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:09:59.396239 24633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:09:59.399089 24633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "23011f367f68410dbe958dbf3bf7e9ad"
format_stamp: "Formatted at 2026-08-07 15:09:59 on dist-test-slave-3kk6"
server_key: "4a02fd2ba1862e397af3445585a96594"
server_key_iv: "dab585cad7c08d7d40df84e8c61d3a45"
server_key_version: "encryptionkey@0"
I20260807 15:09:59.399850 24633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "23011f367f68410dbe958dbf3bf7e9ad"
format_stamp: "Formatted at 2026-08-07 15:09:59 on dist-test-slave-3kk6"
server_key: "4a02fd2ba1862e397af3445585a96594"
server_key_iv: "dab585cad7c08d7d40df84e8c61d3a45"
server_key_version: "encryptionkey@0"
I20260807 15:09:59.404660 24633 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:09:59.407792 24652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:59.409335 24633 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 15:09:59.409528 24633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "23011f367f68410dbe958dbf3bf7e9ad"
format_stamp: "Formatted at 2026-08-07 15:09:59 on dist-test-slave-3kk6"
server_key: "4a02fd2ba1862e397af3445585a96594"
server_key_iv: "dab585cad7c08d7d40df84e8c61d3a45"
server_key_version: "encryptionkey@0"
I20260807 15:09:59.409682 24633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:09:59.450165 24633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:09:59.454411 24633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:09:59.454730 24633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:09:59.455475 24633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:09:59.456501 24633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:09:59.456589 24633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:59.456677 24633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:09:59.456729 24633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:09:59.468065 24633 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:45787
I20260807 15:09:59.468146 24797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:45787 every 8 connection(s)
I20260807 15:09:59.469218 24633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:09:59.478281 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 24633
I20260807 15:09:59.478459 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:09:59.478852 16265 external_mini_cluster.cc:1428] Setting key 6028d7018bac041350d96e7faf834fbe
Aug 07 15:09:59 dist-test-slave-3kk6 krb5kdc[24112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115399, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:09:59.485055 24229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:59.471510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:44933 (local address 127.15.226.126:34387)
0807 15:09:59.471681 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:59.471685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:59.472734 (+ 1049us) server_negotiation.cc:407] Connection header received
0807 15:09:59.474255 (+ 1521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:59.474266 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:59.474335 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:59.474456 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:59.475835 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.476385 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.477562 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.477800 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.477920 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:59.480951 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:59.480955 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:59.480958 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:59.480997 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:59.482582 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:59.483286 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:59.483291 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:59.483292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:59.483397 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:59.483791 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:59.483794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:59.483796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:59.484038 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:59.484166 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:59.484727 (+ 561us) server_negotiation.cc:299] Negotiation successful
0807 15:09:59.484871 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 15:09:59.486033 24801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:59.471958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34387 (local address 127.15.226.67:44933)
0807 15:09:59.472548 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0807 15:09:59.472598 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 15:09:59.474007 (+ 1409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:09:59.474698 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:09:59.474710 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:09:59.475145 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:09:59.475664 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:59.475678 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.476523 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.476528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:59.477427 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:09:59.477437 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.477973 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.477977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:09:59.478181 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:59.478989 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:09:59.479016 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:09:59.480772 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:09:59.482813 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:59.482828 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:59.482844 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:59.483163 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:59.483514 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:09:59.483520 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:09:59.483526 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:09:59.483687 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:09:59.484165 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:09:59.484174 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:09:59.484473 (+ 299us) client_negotiation.cc:769] Sending connection context
0807 15:09:59.484687 (+ 214us) client_negotiation.cc:240] Negotiation successful
0807 15:09:59.485072 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":155,"threads_started":1}
I20260807 15:09:59.487432 24799 heartbeater.cc:347] Connected to a master server at 127.15.226.126:34387
I20260807 15:09:59.487695 24799 heartbeater.cc:464] Registering TS with master...
I20260807 15:09:59.488238 24799 heartbeater.cc:511] Master 127.15.226.126:34387 requested a full tablet report, sending...
I20260807 15:09:59.489482 24178 ts_manager.cc:194] Registered new tserver with Master: 23011f367f68410dbe958dbf3bf7e9ad (127.15.226.67:45787)
I20260807 15:09:59.490104 24178 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.67@KRBTEST.COM'} at 127.15.226.67:44933
I20260807 15:09:59.493224 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/test-admin.keytab.
Aug 07 15:09:59 dist-test-slave-3kk6 krb5kdc[24112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:59.512529 16265 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 15:09:59.515913 24807 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:09:59.514156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48054 (local address 127.15.226.65:38921)
0807 15:09:59.514587 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:59.514596 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:59.514628 (+ 32us) server_negotiation.cc:407] Connection header received
0807 15:09:59.514741 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:59.514750 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:59.515006 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:59.515178 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:59.515744 (+ 566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48054: BlockingRecv error: recv got EOF from 127.0.0.1:48054 (error 108)
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":185,"threads_started":1}
W20260807 15:09:59.516144 16265 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/krb5cc' of type FILE
Aug 07 15:09:59 dist-test-slave-3kk6 krb5kdc[24112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:59.517433 16265 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 15:09:59 dist-test-slave-3kk6 krb5kdc[24112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:59.518550 16265 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 15:09:59 dist-test-slave-3kk6 krb5kdc[24112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.65@KRBTEST.COM
I20260807 15:09:59.529318 24807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:59.519640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48060 (local address 127.15.226.65:38921)
0807 15:09:59.519777 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:59.519781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:59.519800 (+ 19us) server_negotiation.cc:407] Connection header received
0807 15:09:59.519936 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:59.519938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:59.519987 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:59.520067 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:59.520616 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.521511 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.522321 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.522818 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.522948 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:59.525477 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:59.525496 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:59.525535 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:59.525614 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:59.527323 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:59.527913 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:59.527918 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:59.527919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:59.528005 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:59.528263 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:59.528265 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:59.528266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:59.528607 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:59.528894 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:59.529054 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 15:09:59.529190 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 15:09:59.533721 24807 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:09:59.532662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48076 (local address 127.15.226.65:38921)
0807 15:09:59.532802 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:59.532805 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:59.532842 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:09:59.532974 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:59.532976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:59.533027 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:59.533104 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:59.533593 (+ 489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48076: BlockingRecv error: recv got EOF from 127.0.0.1:48076 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260807 15:09:59.533881 16265 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/krb5cc' of type FILE
Aug 07 15:09:59 dist-test-slave-3kk6 krb5kdc[24112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:59.535102 16265 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 15:09:59 dist-test-slave-3kk6 krb5kdc[24112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:59.536087 16265 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 15:09:59 dist-test-slave-3kk6 krb5kdc[24112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.65@KRBTEST.COM
I20260807 15:09:59.546171 24807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:59.537007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48078 (local address 127.15.226.65:38921)
0807 15:09:59.537153 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:59.537157 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:59.537265 (+ 108us) server_negotiation.cc:407] Connection header received
0807 15:09:59.537437 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:59.537441 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:59.537493 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:59.537590 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:59.538149 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.538823 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.539750 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.540019 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.540141 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:59.542757 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:59.542760 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:59.542762 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:59.542790 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:59.544241 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:59.544739 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:59.544744 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:59.544746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:59.544843 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:59.545167 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:59.545172 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:59.545174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:59.545385 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:59.545568 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:59.545835 (+ 267us) server_negotiation.cc:299] Negotiation successful
0807 15:09:59.545985 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260807 15:09:59.549521 24807 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:09:59.548339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48092 (local address 127.15.226.65:38921)
0807 15:09:59.548494 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:59.548498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:59.548577 (+ 79us) server_negotiation.cc:407] Connection header received
0807 15:09:59.548753 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:59.548756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:59.548794 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:59.548857 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:59.549395 (+ 538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48092: BlockingRecv error: recv got EOF from 127.0.0.1:48092 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260807 15:09:59.549676 16265 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786115365131938-16265-0/krb5kdc/krb5cc' of type FILE
Aug 07 15:09:59 dist-test-slave-3kk6 krb5kdc[24112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:59.551012 16265 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 15:09:59 dist-test-slave-3kk6 krb5kdc[24112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:09:59.552024 16265 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 15:09:59 dist-test-slave-3kk6 krb5kdc[24112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.65@KRBTEST.COM
I20260807 15:09:59.562156 24807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:09:59.553162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48098 (local address 127.15.226.65:38921)
0807 15:09:59.553304 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 15:09:59.553308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:09:59.553513 (+ 205us) server_negotiation.cc:407] Connection header received
0807 15:09:59.553592 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:09:59.553594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:09:59.553641 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:09:59.553725 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:09:59.554377 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.555043 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.555928 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:09:59.556179 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:09:59.556278 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:09:59.559059 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:09:59.559063 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:09:59.559065 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:09:59.559095 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:09:59.560597 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:59.561040 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:59.561043 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:59.561045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:59.561130 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:09:59.561395 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:09:59.561398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:09:59.561399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:09:59.561581 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:09:59.561723 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 15:09:59.561910 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 15:09:59.562018 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 15:09:59.563668 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 24262
I20260807 15:09:59.579218 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 24462
I20260807 15:09:59.595784 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 24633
I20260807 15:09:59.607046 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 24128
2026-08-07T15:09:59Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2625 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:09:59 dist-test-slave-3kk6 krb5kdc[24828](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:09:59 dist-test-slave-3kk6 krb5kdc[24828](info): set up 2 sockets
Aug 07 15:09:59 dist-test-slave-3kk6 krb5kdc[24828](info): commencing operation
krb5kdc: starting...
W20260807 15:10:00.889411 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:10:00 dist-test-slave-3kk6 krb5kdc[24828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:10:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:00Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.126.keytab.
Entry for principal oryx/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.126.keytab.
I20260807 15:10:01.034492 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:44957
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.126.keytab
--principal=oryx/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:43293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:44957
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:01.157198 24847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:01.157595 24847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:01.157675 24847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:01.162230 24847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:01.162336 24847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:01.162375 24847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:01.162421 24847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:01.162461 24847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:01.168406 24847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:43293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:44957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.126.keytab
--principal=oryx/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:44957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:01.169688 24847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:01.170851 24847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:10:01.180380 24847 server_base.cc:1034] running on GCE node
W20260807 15:10:01.180310 24857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:01.180310 24853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:01.180310 24855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:01.181303 24847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:01.182765 24847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:01.183991 24847 hybrid_clock.cc:648] HybridClock initialized: now 1786115401183976 us; error 43 us; skew 500 ppm
Aug 07 15:10:01 dist-test-slave-3kk6 krb5kdc[24828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115401, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:01.187147 24847 init.cc:377] Logged in from keytab as oryx/127.15.226.126@KRBTEST.COM (short username oryx)
I20260807 15:10:01.188576 24847 webserver.cc:460] Webserver started at http://127.15.226.126:39731/ using document root <none> and password file <none>
I20260807 15:10:01.189196 24847 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:01.189301 24847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:01.189558 24847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:01.191932 24847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "1be44596da5d4958b056b014ab975a39"
format_stamp: "Formatted at 2026-08-07 15:10:01 on dist-test-slave-3kk6"
server_key: "bc8471a7b4460d97a80b601168f6be9b"
server_key_iv: "9bd67ce600a91e30a87a63c31a9b2b21"
server_key_version: "encryptionkey@0"
I20260807 15:10:01.192580 24847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "1be44596da5d4958b056b014ab975a39"
format_stamp: "Formatted at 2026-08-07 15:10:01 on dist-test-slave-3kk6"
server_key: "bc8471a7b4460d97a80b601168f6be9b"
server_key_iv: "9bd67ce600a91e30a87a63c31a9b2b21"
server_key_version: "encryptionkey@0"
I20260807 15:10:01.196890 24847 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:10:01.199535 24866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:01.200924 24847 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 15:10:01.201092 24847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "1be44596da5d4958b056b014ab975a39"
format_stamp: "Formatted at 2026-08-07 15:10:01 on dist-test-slave-3kk6"
server_key: "bc8471a7b4460d97a80b601168f6be9b"
server_key_iv: "9bd67ce600a91e30a87a63c31a9b2b21"
server_key_version: "encryptionkey@0"
I20260807 15:10:01.201223 24847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:01.214228 24847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:01.217953 24847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:01.218214 24847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:01.226212 24847 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:44957
I20260807 15:10:01.226536 24937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:44957 every 8 connection(s)
I20260807 15:10:01.227404 24847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:01.230500 24938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:01.233395 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 24847
I20260807 15:10:01.233572 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:10:01.233892 16265 external_mini_cluster.cc:1428] Setting key 96ae5b8d9e6c27bd82214a3b42dc94b1
I20260807 15:10:01.236961 24938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39: Bootstrap starting.
I20260807 15:10:01.239907 24938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:01.240880 24938 log.cc:824] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:01.243263 24938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39: No bootstrap required, opened a new log
Aug 07 15:10:01 dist-test-slave-3kk6 krb5kdc[24828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM
I20260807 15:10:01.248037 24938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } }
I20260807 15:10:01.248373 24938 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:01.248446 24938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1be44596da5d4958b056b014ab975a39, State: Initialized, Role: FOLLOWER
I20260807 15:10:01.248963 24938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } }
I20260807 15:10:01.249099 24938 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:01.249192 24938 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:01.249300 24938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:01.249838 24941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:01.235401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43816 (local address 127.15.226.126:44957)
0807 15:10:01.235870 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:01.235886 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:01.235924 (+ 38us) server_negotiation.cc:407] Connection header received
0807 15:10:01.236559 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:01.236576 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:01.236930 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:01.237272 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:01.238238 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:01.239442 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:01.240679 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:01.241273 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:01.241462 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:01.244502 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:01.244508 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:01.244524 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:01.244555 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:01.246412 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:01.246971 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:01.246977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:01.246982 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:01.247105 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:01.247434 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:01.247436 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:01.247437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:01.247872 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:01.248040 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:01.248405 (+ 365us) server_negotiation.cc:299] Negotiation successful
0807 15:10:01.248766 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":149,"threads_started":1}
I20260807 15:10:01.250455 24938 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } }
I20260807 15:10:01.250846 24938 leader_election.cc:302] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1be44596da5d4958b056b014ab975a39; no voters:
I20260807 15:10:01.251236 24938 leader_election.cc:288] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:10:01.251348 24944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:01.251565 24944 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [term 1 LEADER]: Becoming Leader. State: Replica: 1be44596da5d4958b056b014ab975a39, State: Running, Role: LEADER
I20260807 15:10:01.251884 24944 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } }
I20260807 15:10:01.252657 24938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:01.253153 24944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1be44596da5d4958b056b014ab975a39. Latest consensus state: current_term: 1 leader_uuid: "1be44596da5d4958b056b014ab975a39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } }
I20260807 15:10:01.253284 24944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:01.253620 24948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1be44596da5d4958b056b014ab975a39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } }
I20260807 15:10:01.253758 24948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:01.253865 24950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:01.257256 24950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:10:01.264148 24950 catalog_manager.cc:1347] Generated new cluster ID: 3115c820c8e740de80daae454f6c5f1c
I20260807 15:10:01.264252 24950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:10:01.282307 24950 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:10:01.283435 24950 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:01.290247 24950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39: Generated new TSK 0
I20260807 15:10:01.291042 24950 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.226.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.226.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.226.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.65.keytab.
Entry for principal oryx/127.15.226.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.65.keytab.
I20260807 15:10:01.330565 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:44957
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.65.keytab
--principal=oryx/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:43293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:01.481266 24973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:01.481611 24973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:01.481663 24973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:01.486215 24973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:01.486323 24973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:01.486433 24973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:10:01.492262 24973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:43293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.65.keytab
--principal=oryx/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:44957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:01.493536 24973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:01.494701 24973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:01.505343 24978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:01.505358 24982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:01.505354 24979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:01.505458 24973 server_base.cc:1034] running on GCE node
I20260807 15:10:01.506206 24973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:01.507030 24973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:01.508241 24973 hybrid_clock.cc:648] HybridClock initialized: now 1786115401508198 us; error 62 us; skew 500 ppm
Aug 07 15:10:01 dist-test-slave-3kk6 krb5kdc[24828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115401, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:01.511430 24973 init.cc:377] Logged in from keytab as oryx/127.15.226.65@KRBTEST.COM (short username oryx)
I20260807 15:10:01.512477 24973 webserver.cc:460] Webserver started at http://127.15.226.65:45699/ using document root <none> and password file <none>
I20260807 15:10:01.513046 24973 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:01.513108 24973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:01.513285 24973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:01.515563 24973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "60340eaffc12444aaf7a470a0634f9ac"
format_stamp: "Formatted at 2026-08-07 15:10:01 on dist-test-slave-3kk6"
server_key: "d0573ff99f213938b9f0fb9ee7ccfdcd"
server_key_iv: "f65caad6614b6b6c1aaa01c769925ca2"
server_key_version: "encryptionkey@0"
I20260807 15:10:01.516146 24973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "60340eaffc12444aaf7a470a0634f9ac"
format_stamp: "Formatted at 2026-08-07 15:10:01 on dist-test-slave-3kk6"
server_key: "d0573ff99f213938b9f0fb9ee7ccfdcd"
server_key_iv: "f65caad6614b6b6c1aaa01c769925ca2"
server_key_version: "encryptionkey@0"
I20260807 15:10:01.520332 24973 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:10:01.523058 24995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:01.524439 24973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 15:10:01.524556 24973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "60340eaffc12444aaf7a470a0634f9ac"
format_stamp: "Formatted at 2026-08-07 15:10:01 on dist-test-slave-3kk6"
server_key: "d0573ff99f213938b9f0fb9ee7ccfdcd"
server_key_iv: "f65caad6614b6b6c1aaa01c769925ca2"
server_key_version: "encryptionkey@0"
I20260807 15:10:01.524665 24973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:01.536463 24973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:01.540282 24973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:01.540464 24973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:01.541132 24973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:01.541970 24973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:01.542021 24973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:01.542063 24973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:01.542078 24973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:01.552609 24973 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:37605
I20260807 15:10:01.552639 25141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:37605 every 8 connection(s)
I20260807 15:10:01.553669 24973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:10:01.560755 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 24973
I20260807 15:10:01.560920 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:10:01.561247 16265 external_mini_cluster.cc:1428] Setting key fa7d15d3b50b131293dad1b4cde6d7e7
Aug 07 15:10:01 dist-test-slave-3kk6 krb5kdc[24828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115401, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM
I20260807 15:10:01.569023 24941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:01.555815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:53475 (local address 127.15.226.126:44957)
0807 15:10:01.556010 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:01.556015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:01.556869 (+ 854us) server_negotiation.cc:407] Connection header received
0807 15:10:01.558284 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:01.558287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:01.558340 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:01.558424 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:01.559761 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:01.560320 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:01.561624 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:01.561880 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:01.561993 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:01.565037 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:01.565043 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:01.565052 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:01.565077 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:01.566730 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:01.567303 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:01.567306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:01.567308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:01.567401 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:01.567790 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:01.567794 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:01.567795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:01.568007 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:01.568122 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:01.568692 (+ 570us) server_negotiation.cc:299] Negotiation successful
0807 15:10:01.568835 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 15:10:01.569958 25145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:01.556207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.65:53475)
0807 15:10:01.556699 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:01.556746 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:01.558058 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:01.558607 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:01.558614 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:01.559078 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:01.559616 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:01.559631 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:01.560439 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:01.560441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:01.561491 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:01.561502 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:01.562063 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:01.562068 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:01.562237 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:01.563022 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:01.563040 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:01.564850 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:01.566859 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:01.566866 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:01.566884 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:01.567181 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:01.567510 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:01.567513 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:01.567515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:01.567683 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:01.568140 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:01.568148 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:01.568436 (+ 288us) client_negotiation.cc:769] Sending connection context
0807 15:10:01.568648 (+ 212us) client_negotiation.cc:240] Negotiation successful
0807 15:10:01.569028 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":127,"threads_started":1}
I20260807 15:10:01.571431 25142 heartbeater.cc:347] Connected to a master server at 127.15.226.126:44957
I20260807 15:10:01.571730 25142 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for oryx/127.15.226.66@KRBTEST.COM; defaulting to no policy
I20260807 15:10:01.572338 25142 heartbeater.cc:511] Master 127.15.226.126:44957 requested a full tablet report, sending...
I20260807 15:10:01.574009 24892 ts_manager.cc:194] Registered new tserver with Master: 60340eaffc12444aaf7a470a0634f9ac (127.15.226.65:37605)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.226.66@KRBTEST.COM" created.
I20260807 15:10:01.575502 24892 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.226.65@KRBTEST.COM'} at 127.15.226.65:53475
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.226.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.66.keytab.
Entry for principal oryx/127.15.226.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.66.keytab.
I20260807 15:10:01.594581 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:44957
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.66.keytab
--principal=oryx/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:43293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:01.716729 25150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:01.717070 25150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:01.717118 25150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:01.721457 25150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:01.721553 25150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:01.721619 25150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:10:01.727531 25150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:43293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.66.keytab
--principal=oryx/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:44957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:01.728832 25150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:01.730063 25150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:01.740738 25159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:01.740703 25158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:01.741008 25150 server_base.cc:1034] running on GCE node
W20260807 15:10:01.740695 25162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:01.741614 25150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:01.742365 25150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:01.743547 25150 hybrid_clock.cc:648] HybridClock initialized: now 1786115401743468 us; error 94 us; skew 500 ppm
Aug 07 15:10:01 dist-test-slave-3kk6 krb5kdc[24828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115401, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:01.746860 25150 init.cc:377] Logged in from keytab as oryx/127.15.226.66@KRBTEST.COM (short username oryx)
I20260807 15:10:01.747888 25150 webserver.cc:460] Webserver started at http://127.15.226.66:34537/ using document root <none> and password file <none>
I20260807 15:10:01.748526 25150 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:01.748631 25150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:01.748888 25150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:01.751252 25150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "f448c8ea9dc5445496f57727ac1061bb"
format_stamp: "Formatted at 2026-08-07 15:10:01 on dist-test-slave-3kk6"
server_key: "12f48013d1d9cff85801c5fdf5842ad5"
server_key_iv: "6ded28c4e262fb5d179925d4fe646381"
server_key_version: "encryptionkey@0"
I20260807 15:10:01.751928 25150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "f448c8ea9dc5445496f57727ac1061bb"
format_stamp: "Formatted at 2026-08-07 15:10:01 on dist-test-slave-3kk6"
server_key: "12f48013d1d9cff85801c5fdf5842ad5"
server_key_iv: "6ded28c4e262fb5d179925d4fe646381"
server_key_version: "encryptionkey@0"
I20260807 15:10:01.756665 25150 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:10:01.759752 25173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:01.761220 25150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:10:01.761406 25150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "f448c8ea9dc5445496f57727ac1061bb"
format_stamp: "Formatted at 2026-08-07 15:10:01 on dist-test-slave-3kk6"
server_key: "12f48013d1d9cff85801c5fdf5842ad5"
server_key_iv: "6ded28c4e262fb5d179925d4fe646381"
server_key_version: "encryptionkey@0"
I20260807 15:10:01.761567 25150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:01.780045 25150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:01.783967 25150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:01.784212 25150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:01.784906 25150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:01.785887 25150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:01.785986 25150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:01.786080 25150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:01.786125 25150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:01.796852 25150 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:44821
I20260807 15:10:01.796922 25327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:44821 every 8 connection(s)
I20260807 15:10:01.797885 25150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:10:01.803853 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 25150
I20260807 15:10:01.803968 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:10:01.804261 16265 external_mini_cluster.cc:1428] Setting key 38deaa39fbf3e5d2722befd7dfae00ff
Aug 07 15:10:01 dist-test-slave-3kk6 krb5kdc[24828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115401, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM
I20260807 15:10:01.814272 24941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:01.800035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:47393 (local address 127.15.226.126:44957)
0807 15:10:01.800205 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:01.800210 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:01.801198 (+ 988us) server_negotiation.cc:407] Connection header received
0807 15:10:01.802591 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:01.802594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:01.802693 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:01.802785 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:01.804071 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:01.804867 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:01.806339 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:01.806584 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:01.806714 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:01.809632 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:01.809638 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:01.809644 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:01.809679 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:01.811732 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:01.812346 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:01.812353 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:01.812358 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:01.812466 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:01.812924 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:01.812931 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:01.812934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:01.813239 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:01.813355 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:01.813897 (+ 542us) server_negotiation.cc:299] Negotiation successful
0807 15:10:01.814050 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 15:10:01.815173 25330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:01.800512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.66:47393)
0807 15:10:01.801013 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:01.801062 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:01.802377 (+ 1315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:01.802940 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:01.802947 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:01.803349 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:01.803882 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:01.803899 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:01.805008 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:01.805011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:01.806195 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:01.806207 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:01.806787 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:01.806792 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:01.806969 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:01.807667 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:01.807684 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:01.809442 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:01.811877 (+ 2435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:01.811883 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:01.811898 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:01.812208 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:01.812641 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:01.812643 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:01.812645 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:01.812812 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:01.813361 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:01.813367 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:01.813631 (+ 264us) client_negotiation.cc:769] Sending connection context
0807 15:10:01.813803 (+ 172us) client_negotiation.cc:240] Negotiation successful
0807 15:10:01.814114 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":126,"threads_started":1}
WARNING: no policy specified for oryx/127.15.226.67@KRBTEST.COM; defaulting to no policy
I20260807 15:10:01.816624 25328 heartbeater.cc:347] Connected to a master server at 127.15.226.126:44957
I20260807 15:10:01.816929 25328 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:01.817605 25328 heartbeater.cc:511] Master 127.15.226.126:44957 requested a full tablet report, sending...
I20260807 15:10:01.819043 24892 ts_manager.cc:194] Registered new tserver with Master: f448c8ea9dc5445496f57727ac1061bb (127.15.226.66:44821)
I20260807 15:10:01.819734 24892 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.226.66@KRBTEST.COM'} at 127.15.226.66:47393
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.226.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.226.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.67.keytab.
Entry for principal oryx/127.15.226.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.67.keytab.
I20260807 15:10:01.839401 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:44957
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.67.keytab
--principal=oryx/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:43293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:01.963477 25334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:01.963845 25334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:01.963955 25334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:01.968238 25334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:01.968341 25334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:01.968439 25334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:10:01.974030 25334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:43293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.67.keytab
--principal=oryx/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:44957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:01.975324 25334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:01.976435 25334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:01.986667 25345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:01.987015 25334 server_base.cc:1034] running on GCE node
W20260807 15:10:01.986645 25343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:01.986672 25339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:01.987627 25334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:01.988351 25334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:01.989555 25334 hybrid_clock.cc:648] HybridClock initialized: now 1786115401989538 us; error 34 us; skew 500 ppm
Aug 07 15:10:01 dist-test-slave-3kk6 krb5kdc[24828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115401, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:01.993185 25334 init.cc:377] Logged in from keytab as oryx/127.15.226.67@KRBTEST.COM (short username oryx)
I20260807 15:10:01.994192 25334 webserver.cc:460] Webserver started at http://127.15.226.67:46813/ using document root <none> and password file <none>
I20260807 15:10:01.994894 25334 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:01.994967 25334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:01.995203 25334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:01.997695 25334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "b85d9bf8304749cea743efe6d297ea65"
format_stamp: "Formatted at 2026-08-07 15:10:01 on dist-test-slave-3kk6"
server_key: "c2e4ab65e424f13ff03a3bd1f9723104"
server_key_iv: "fb7486795026f044877c0c359b1cb4dc"
server_key_version: "encryptionkey@0"
I20260807 15:10:01.998370 25334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "b85d9bf8304749cea743efe6d297ea65"
format_stamp: "Formatted at 2026-08-07 15:10:01 on dist-test-slave-3kk6"
server_key: "c2e4ab65e424f13ff03a3bd1f9723104"
server_key_iv: "fb7486795026f044877c0c359b1cb4dc"
server_key_version: "encryptionkey@0"
I20260807 15:10:02.002562 25334 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 15:10:02.005527 25354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:02.007247 25334 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 15:10:02.007400 25334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "b85d9bf8304749cea743efe6d297ea65"
format_stamp: "Formatted at 2026-08-07 15:10:01 on dist-test-slave-3kk6"
server_key: "c2e4ab65e424f13ff03a3bd1f9723104"
server_key_iv: "fb7486795026f044877c0c359b1cb4dc"
server_key_version: "encryptionkey@0"
I20260807 15:10:02.007537 25334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:02.020037 25334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:02.023893 25334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:02.024137 25334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:02.024829 25334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:02.025759 25334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:02.025808 25334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:02.025875 25334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:02.025913 25334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:02.035706 25334 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:43089
I20260807 15:10:02.035742 25511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:43089 every 8 connection(s)
I20260807 15:10:02.036697 25334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:10:02.038245 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 25334
I20260807 15:10:02.038403 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:10:02.038795 16265 external_mini_cluster.cc:1428] Setting key e8ce814fce0edb15da1011fbd3581b2e
Aug 07 15:10:02 dist-test-slave-3kk6 krb5kdc[24828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115401, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM
I20260807 15:10:02.052994 24941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.038987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:47701 (local address 127.15.226.126:44957)
0807 15:10:02.039176 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:02.039181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:02.040322 (+ 1141us) server_negotiation.cc:407] Connection header received
0807 15:10:02.042048 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:02.042051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:02.042114 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:02.042209 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:02.043513 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.044072 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.045293 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.045522 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.045620 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.048582 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:02.048586 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:02.048588 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:02.048617 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:02.050302 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.050918 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.050921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.050922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.051008 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.051450 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.051453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.051454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.051680 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:02.051844 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:02.052589 (+ 745us) server_negotiation.cc:299] Negotiation successful
0807 15:10:02.052726 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 15:10:02.054132 25516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.039488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.67:47701)
0807 15:10:02.040133 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:02.040207 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:02.041803 (+ 1596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:02.042378 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:02.042386 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:02.042816 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:02.043337 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.043351 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.044194 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.044197 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.045150 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.045160 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.045685 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.045688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.045850 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.046593 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:02.046610 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:02.048367 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:02.050450 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.050455 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.050470 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.050804 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.051127 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.051130 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.051131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.051300 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.051813 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:02.051819 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:02.052223 (+ 404us) client_negotiation.cc:769] Sending connection context
0807 15:10:02.052452 (+ 229us) client_negotiation.cc:240] Negotiation successful
0807 15:10:02.052908 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":163,"threads_started":1}
I20260807 15:10:02.055402 25513 heartbeater.cc:347] Connected to a master server at 127.15.226.126:44957
I20260807 15:10:02.055680 25513 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:02.056257 25513 heartbeater.cc:511] Master 127.15.226.126:44957 requested a full tablet report, sending...
I20260807 15:10:02.057423 24892 ts_manager.cc:194] Registered new tserver with Master: b85d9bf8304749cea743efe6d297ea65 (127.15.226.67:43089)
I20260807 15:10:02.058050 24892 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.226.67@KRBTEST.COM'} at 127.15.226.67:47701
I20260807 15:10:02.066224 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.226.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.226.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.226.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.125.keytab.
Entry for principal oryx/127.15.226.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.125.keytab.
I20260807 15:10:02.100667 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.125:45651
--webserver_interface=127.15.226.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.125.keytab
--principal=oryx/127.15.226.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:43293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:44957,127.15.226.125:45651
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:02.234934 25521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:02.235255 25521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:02.235298 25521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:02.239856 25521 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:02.239943 25521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:02.239964 25521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:02.239984 25521 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:02.240000 25521 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:02.246999 25521 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:43293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:44957,127.15.226.125:45651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.125.keytab
--principal=oryx/127.15.226.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.125:45651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.226.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
W20260807 15:10:02.247295 25521 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.15.226.126:44957,127.15.226.125:45651'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 15:10:02.248315 25521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:02.249423 25521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:02.259542 25536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:02.259721 25531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:02.259806 25532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:02.259943 25521 server_base.cc:1034] running on GCE node
I20260807 15:10:02.260635 25521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:02.261888 25521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:02.263104 25521 hybrid_clock.cc:648] HybridClock initialized: now 1786115402263090 us; error 40 us; skew 500 ppm
Aug 07 15:10:02 dist-test-slave-3kk6 krb5kdc[24828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115402, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:02.266361 25521 init.cc:377] Logged in from keytab as oryx/127.15.226.125@KRBTEST.COM (short username oryx)
I20260807 15:10:02.267699 25521 webserver.cc:460] Webserver started at http://127.15.226.125:36197/ using document root <none> and password file <none>
I20260807 15:10:02.268359 25521 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:02.268467 25521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:02.268738 25521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:02.271286 25521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/data/instance:
uuid: "582deee640a1481cbf7a09c88c6269f7"
format_stamp: "Formatted at 2026-08-07 15:10:02 on dist-test-slave-3kk6"
server_key: "d08f0988317217e8089b70cab3122665"
server_key_iv: "500e1415e0beaaf37da1abaf1afa0ee6"
server_key_version: "encryptionkey@0"
I20260807 15:10:02.272001 25521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/wal/instance:
uuid: "582deee640a1481cbf7a09c88c6269f7"
format_stamp: "Formatted at 2026-08-07 15:10:02 on dist-test-slave-3kk6"
server_key: "d08f0988317217e8089b70cab3122665"
server_key_iv: "500e1415e0beaaf37da1abaf1afa0ee6"
server_key_version: "encryptionkey@0"
I20260807 15:10:02.276437 25521 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:10:02.279512 25544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:02.281015 25521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:10:02.281198 25521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/wal
uuid: "582deee640a1481cbf7a09c88c6269f7"
format_stamp: "Formatted at 2026-08-07 15:10:02 on dist-test-slave-3kk6"
server_key: "d08f0988317217e8089b70cab3122665"
server_key_iv: "500e1415e0beaaf37da1abaf1afa0ee6"
server_key_version: "encryptionkey@0"
I20260807 15:10:02.281338 25521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:02.328177 25521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:02.332376 25521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:02.332621 25521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:02.341624 25521 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.125:45651
I20260807 15:10:02.342051 25608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.125:45651 every 8 connection(s)
I20260807 15:10:02.342971 25521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/data/info.pb
I20260807 15:10:02.346125 25610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:02.350483 25610 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:10:02.351050 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 25521
Aug 07 15:10:02 dist-test-slave-3kk6 krb5kdc[24828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115402, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.125@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM
Aug 07 15:10:02 dist-test-slave-3kk6 krb5kdc[24828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.226.125@KRBTEST.COM
I20260807 15:10:02.366978 24941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.352920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43832 (local address 127.15.226.126:44957)
0807 15:10:02.353080 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:02.353085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:02.354186 (+ 1101us) server_negotiation.cc:407] Connection header received
0807 15:10:02.355338 (+ 1152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:02.355342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:02.355432 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:02.355544 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:02.356973 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.357772 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.359063 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.359322 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.359447 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.362557 (+ 3110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:02.362560 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:02.362563 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:02.362599 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:02.364413 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.365019 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.365022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.365023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.365110 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.365528 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.365532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.365533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.365903 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:02.366036 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:02.366571 (+ 535us) server_negotiation.cc:299] Negotiation successful
0807 15:10:02.366781 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 15:10:02.368045 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.353102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.0.0.1:43832)
0807 15:10:02.353993 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:02.354071 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:02.355017 (+ 946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:02.355758 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:02.355764 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:02.356310 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:02.356731 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.356751 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.357946 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.357949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.358909 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.358920 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.359520 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.359525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.359661 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.360524 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:02.360541 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:02.362348 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:02.364546 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.364558 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.364570 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.364906 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.365223 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.365229 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.365231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.365398 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.366048 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:02.366055 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:02.366282 (+ 227us) client_negotiation.cc:769] Sending connection context
0807 15:10:02.366501 (+ 219us) client_negotiation.cc:240] Negotiation successful
0807 15:10:02.366987 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"spinlock_wait_cycles":31104,"thread_start_us":250,"threads_started":1}
I20260807 15:10:02.368045 25613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.353103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35022 (local address 127.15.226.125:45651)
0807 15:10:02.353865 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:02.353888 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:02.353932 (+ 44us) server_negotiation.cc:407] Connection header received
0807 15:10:02.355001 (+ 1069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:02.355023 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:02.355338 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:02.355544 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:02.356615 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.358058 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.359036 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.359465 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.359661 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.363109 (+ 3448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:02.363116 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:02.363129 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:02.363152 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:02.364732 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.365272 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.365278 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.365279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.365392 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.365703 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.365707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.365708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.366264 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:02.366429 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:02.366752 (+ 323us) server_negotiation.cc:299] Negotiation successful
0807 15:10:02.367181 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":97,"server-negotiator.queue_time_us":598,"thread_start_us":194,"threads_started":1}
I20260807 15:10:02.369590 25610 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:10:02 dist-test-slave-3kk6 krb5kdc[24828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115402, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.125@KRBTEST.COM for oryx/127.15.226.125@KRBTEST.COM
I20260807 15:10:02.378849 25613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.370354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35032 (local address 127.15.226.125:45651)
0807 15:10:02.370473 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:02.370480 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:02.370499 (+ 19us) server_negotiation.cc:407] Connection header received
0807 15:10:02.370531 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:02.370533 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:02.370574 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:02.370656 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:02.371099 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.372021 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.372824 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.373058 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.373171 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.375670 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:02.375673 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:02.375675 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:02.375696 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:02.377053 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.377497 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.377507 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.377508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.377588 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.377879 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.377883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.377884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.378100 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:02.378209 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:02.378477 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 15:10:02.378639 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":50}
I20260807 15:10:02.378849 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.370173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:45651 (local address 127.0.0.1:35032)
0807 15:10:02.370301 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:02.370319 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:02.370429 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:02.370704 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:02.370707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:02.370893 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:02.370982 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.370988 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.372140 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.372146 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.372691 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.372699 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.373169 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.373172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.373295 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.373860 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:02.373892 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:02.375511 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:02.377173 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.377178 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.377180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.377388 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.377684 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.377689 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.377691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.377778 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.378209 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:02.378212 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:02.378308 (+ 96us) client_negotiation.cc:769] Sending connection context
0807 15:10:02.378417 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 15:10:02.378586 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":21}
I20260807 15:10:02.382294 25610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7: Bootstrap starting.
I20260807 15:10:02.385023 25610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:02.385917 25610 log.cc:824] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:02.388451 25610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7: No bootstrap required, opened a new log
I20260807 15:10:02.392081 25610 raft_consensus.cc:348] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } }
I20260807 15:10:02.392372 25610 raft_consensus.cc:374] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:02.392427 25610 raft_consensus.cc:729] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 582deee640a1481cbf7a09c88c6269f7, State: Initialized, Role: FOLLOWER
I20260807 15:10:02.392946 25610 consensus_queue.cc:260] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } }
I20260807 15:10:02.393505 25616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } } }
I20260807 15:10:02.393651 25616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:10:02.394411 25610 sys_catalog.cc:564] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 15:10:02.399822 25634 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:10:02.399937 25634 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:10:02.402905 25521 master_runner.cc:428] Detected that this master 582deee640a1481cbf7a09c88c6269f7 is joining an existing cluster
I20260807 15:10:02.402976 25521 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.226.125:45651
I20260807 15:10:02.414706 25646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.405428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:45651 (local address 127.0.0.1:35046)
0807 15:10:02.405785 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:02.405800 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:02.405909 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:02.406204 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:02.406207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:02.406528 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:02.406729 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.406738 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.407557 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.407562 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.408345 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.408355 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.409003 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.409007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.409103 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.409792 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:02.409804 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:02.410216 (+ 412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:02.412500 (+ 2284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.412503 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.412506 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.413088 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.413446 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.413450 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.413452 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.413553 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.414054 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:02.414056 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:02.414136 (+ 80us) client_negotiation.cc:769] Sending connection context
0807 15:10:02.414238 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 15:10:02.414401 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":31,"thread_start_us":123,"threads_started":1}
I20260807 15:10:02.414709 25613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.405740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35046 (local address 127.15.226.125:45651)
0807 15:10:02.405854 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:02.405861 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:02.405875 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:10:02.406048 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:02.406052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:02.406100 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:02.406185 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:02.406867 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.407431 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.408480 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.408877 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.408991 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.410372 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:02.410376 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:02.410379 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:02.410413 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:02.412370 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.413206 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.413212 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.413216 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.413320 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.413654 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.413658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.413659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.413921 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:02.414019 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:02.414306 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 15:10:02.414483 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":42}
I20260807 15:10:02.415410 24941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.406001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43834 (local address 127.15.226.126:44957)
0807 15:10:02.406164 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:02.406171 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:02.406187 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:10:02.406229 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:02.406232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:02.406277 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:02.406353 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:02.407011 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.407854 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.408905 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.409250 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.409368 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.411768 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:02.411775 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:02.411779 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:02.411813 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:02.413532 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.413961 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.413965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.413967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.414064 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.414375 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.414380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.414385 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.414642 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:02.414749 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:02.415098 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 15:10:02.415242 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 15:10:02.415899 25645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.405327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.0.0.1:43834)
0807 15:10:02.405649 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:02.405669 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:02.405778 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:02.406364 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:02.406367 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:02.406638 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:02.406859 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.406866 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.407989 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.407993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.408780 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.408789 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.409395 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.409400 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.410258 (+ 858us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.410974 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:02.410989 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:02.411560 (+ 571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:02.413642 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.413647 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.413649 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.413855 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.414158 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.414163 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.414165 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.414260 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.414829 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:02.414832 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:02.414917 (+ 85us) client_negotiation.cc:769] Sending connection context
0807 15:10:02.415563 (+ 646us) client_negotiation.cc:240] Negotiation successful
0807 15:10:02.415717 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":33,"thread_start_us":139,"threads_started":1}
Aug 07 15:10:02 dist-test-slave-3kk6 krb5kdc[24828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115401, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.126@KRBTEST.COM for oryx/127.15.226.125@KRBTEST.COM
I20260807 15:10:02.428860 25616 raft_consensus.cc:482] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:10:02.429085 25616 raft_consensus.cc:504] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } }
I20260807 15:10:02.430156 25616 leader_election.cc:288] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1be44596da5d4958b056b014ab975a39 (127.15.226.126:44957)
I20260807 15:10:02.430763 24905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "582deee640a1481cbf7a09c88c6269f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1be44596da5d4958b056b014ab975a39" is_pre_election: true
I20260807 15:10:02.431576 25548 leader_election.cc:302] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 582deee640a1481cbf7a09c88c6269f7; no voters: 1be44596da5d4958b056b014ab975a39
I20260807 15:10:02.431910 25616 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
WARNING: no policy specified for oryx/127.15.226.124@KRBTEST.COM; defaulting to no policy
I20260807 15:10:02.435755 25613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.421977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35062 (local address 127.15.226.125:45651)
0807 15:10:02.422131 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:02.422136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:02.422152 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:10:02.422222 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:02.422225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:02.422279 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:02.422362 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:02.423446 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.424247 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.425655 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.426069 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.426205 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.431756 (+ 5551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:02.431760 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:02.431763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:02.431795 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:02.433630 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.434213 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.434217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.434219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.434360 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.434697 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.434704 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.434706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.434929 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:02.435049 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:02.435440 (+ 391us) server_negotiation.cc:299] Negotiation successful
0807 15:10:02.435578 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 15:10:02.435810 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.421121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:45651 (local address 127.0.0.1:35062)
0807 15:10:02.421539 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:02.421594 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:02.421729 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:02.422539 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:02.422546 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:02.423107 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:02.423280 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.423290 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.424420 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.424423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.425501 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.425515 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.426225 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.426229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.426379 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.427177 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:02.427194 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:02.431552 (+ 4358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:02.433782 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.433797 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.433804 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.434088 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.434464 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.434468 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.434470 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.434572 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.435053 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:02.435064 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:02.435210 (+ 146us) client_negotiation.cc:769] Sending connection context
0807 15:10:02.435379 (+ 169us) client_negotiation.cc:240] Negotiation successful
0807 15:10:02.435626 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":156,"threads_started":1}
I20260807 15:10:02.437502 24892 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.226.125:45651: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "582deee640a1481cbf7a09c88c6269f7"
member_type: NON_VOTER
last_known_addr {
host: "127.15.226.125"
port: 45651
}
attrs {
promote: true
}
}
dest_uuid: "1be44596da5d4958b056b014ab975a39"
cas_config_opid_index: -1
I20260807 15:10:02.438417 24892 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: NON_VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.226.124@KRBTEST.COM" created.
I20260807 15:10:02.441442 25576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:02.442847 25576 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 1 FOLLOWER]: Refusing update from remote peer 1be44596da5d4958b056b014ab975a39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 15:10:02.440816 25651 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 582deee640a1481cbf7a09c88c6269f7 (127.15.226.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: NON_VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: true } } }
I20260807 15:10:02.443662 25650 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: NON_VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:02.444947 25651 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.226.125:45651
I20260807 15:10:02.445566 25521 master.cc:537] Master@127.15.226.125:45651 shutting down...
I20260807 15:10:02.445537 25651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1be44596da5d4958b056b014ab975a39" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: NON_VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: true } } }
I20260807 15:10:02.446069 25651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [sys.catalog]: This master's current role is: LEADER
W20260807 15:10:02.448082 24868 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260807 15:10:02.449272 24868 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 -> Peer 582deee640a1481cbf7a09c88c6269f7 (127.15.226.125:45651): Couldn't send request to peer 582deee640a1481cbf7a09c88c6269f7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 15:10:02.451713 25521 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:10:02.452147 25521 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:02.452308 25521 tablet_replica.cc:330] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.226.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.124.keytab.
Entry for principal oryx/127.15.226.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.124.keytab.
I20260807 15:10:02.462302 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.124:46831
--webserver_interface=127.15.226.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.124.keytab
--principal=oryx/127.15.226.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:43293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:44957,127.15.226.125:45651,127.15.226.124:46831
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:10:02.469091 25521 master.cc:552] Master@127.15.226.125:45651 shutdown complete.
I20260807 15:10:02.469173 25521 master_runner.cc:305] Clearing existing system tablet
I20260807 15:10:02.469452 25521 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:02.469519 25521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:02.471081 25521 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 15:10:02.472549 25662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:02.472903 25521 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:02.473039 25521 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 15:10:02.473074 25521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/wal
uuid: "582deee640a1481cbf7a09c88c6269f7"
format_stamp: "Formatted at 2026-08-07 15:10:02 on dist-test-slave-3kk6"
server_key: "d08f0988317217e8089b70cab3122665"
server_key_iv: "500e1415e0beaaf37da1abaf1afa0ee6"
server_key_version: "encryptionkey@0"
I20260807 15:10:02.475545 25521 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:02.478300 25521 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 15:10:02.478416 25521 log.cc:1196] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 15:10:02.478744 25521 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7: Deleting consensus metadata
I20260807 15:10:02.478942 25521 master_runner.cc:315] Copying system tablet from 127.15.226.126:44957
I20260807 15:10:02.480458 25521 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7: tablet copy: Beginning tablet copy session from remote peer at address 127.15.226.126:44957
I20260807 15:10:02.491118 25672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.481295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.0.0.1:43836)
0807 15:10:02.481675 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:02.481705 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:02.481851 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:02.482171 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:02.482174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:02.482415 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:02.482560 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.482568 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.483929 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.483933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.484854 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.484867 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.485534 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.485538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.485682 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.486388 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:02.486405 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:02.486941 (+ 536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:02.489202 (+ 2261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.489206 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.489209 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.489495 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.489834 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.489837 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.489839 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.489946 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.490424 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:02.490428 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:02.490541 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 15:10:02.490710 (+ 169us) client_negotiation.cc:240] Negotiation successful
0807 15:10:02.490881 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":105,"threads_started":1}
I20260807 15:10:02.491734 24941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.481365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43836 (local address 127.15.226.126:44957)
0807 15:10:02.481510 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:02.481516 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:02.481825 (+ 309us) server_negotiation.cc:407] Connection header received
0807 15:10:02.481962 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:02.481966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:02.482029 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:02.482126 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:02.482868 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.483780 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.485007 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.485389 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.485513 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.487122 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:02.487126 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:02.487129 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:02.487162 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:02.489060 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.489611 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.489615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.489616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.489722 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.490051 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.490055 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.490057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.490301 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:02.490415 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:02.491091 (+ 676us) server_negotiation.cc:299] Negotiation successful
0807 15:10:02.491556 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 15:10:02.492720 24917 tablet_copy_service.cc:140] P 1be44596da5d4958b056b014ab975a39: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 582deee640a1481cbf7a09c88c6269f7 ({username='oryx', principal='oryx/127.15.226.125@KRBTEST.COM'} at 127.0.0.1:43836)
I20260807 15:10:02.492893 24917 tablet_copy_service.cc:161] P 1be44596da5d4958b056b014ab975a39: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 582deee640a1481cbf7a09c88c6269f7 at {username='oryx', principal='oryx/127.15.226.125@KRBTEST.COM'} at 127.0.0.1:43836: session id = 582deee640a1481cbf7a09c88c6269f7-00000000000000000000000000000000
I20260807 15:10:02.494944 24917 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 15:10:02.497113 25521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 15:10:02.500048 25521 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7: tablet copy: Starting download of 0 data blocks...
I20260807 15:10:02.500285 25521 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7: tablet copy: Starting download of 1 WAL segments...
I20260807 15:10:02.502550 25521 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 15:10:02.505043 24917 tablet_copy_service.cc:342] P 1be44596da5d4958b056b014ab975a39: Request end of tablet copy session 582deee640a1481cbf7a09c88c6269f7-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.226.125@KRBTEST.COM'} at 127.0.0.1:43836
I20260807 15:10:02.505200 24917 tablet_copy_service.cc:433] P 1be44596da5d4958b056b014ab975a39: ending tablet copy session 582deee640a1481cbf7a09c88c6269f7-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 582deee640a1481cbf7a09c88c6269f7
I20260807 15:10:02.514806 25521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:02.516665 25673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:02.516724 25677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:02.517680 25521 server_base.cc:1034] running on GCE node
W20260807 15:10:02.518851 25674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:02.519141 25521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:02.519635 25521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:02.520824 25521 hybrid_clock.cc:648] HybridClock initialized: now 1786115402520812 us; error 38 us; skew 500 ppm
Aug 07 15:10:02 dist-test-slave-3kk6 krb5kdc[24828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115402, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:02.522782 25521 init.cc:377] Logged in from keytab as oryx/127.15.226.125@KRBTEST.COM (short username oryx)
I20260807 15:10:02.523399 25521 webserver.cc:460] Webserver started at http://127.15.226.125:33351/ using document root <none> and password file <none>
I20260807 15:10:02.523624 25521 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:02.523674 25521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:02.525226 25521 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:10:02.526379 25688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:02.526748 25521 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:10:02.526836 25521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/wal
uuid: "582deee640a1481cbf7a09c88c6269f7"
format_stamp: "Formatted at 2026-08-07 15:10:02 on dist-test-slave-3kk6"
server_key: "d08f0988317217e8089b70cab3122665"
server_key_iv: "500e1415e0beaaf37da1abaf1afa0ee6"
server_key_version: "encryptionkey@0"
I20260807 15:10:02.526933 25521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:02.562436 25521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:02.565125 25521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:02.571552 25521 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.125:45651
I20260807 15:10:02.571576 25760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.125:45651 every 8 connection(s)
I20260807 15:10:02.572324 25521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-1/data/info.pb
I20260807 15:10:02.574602 25761 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:10:02.576035 25761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7: Bootstrap starting.
I20260807 15:10:02.578639 25142 heartbeater.cc:503] Master 127.15.226.126:44957 was elected leader, sending a full tablet report...
I20260807 15:10:02.590001 25761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:10:02.590727 25761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7: Bootstrap complete.
I20260807 15:10:02.591457 25761 raft_consensus.cc:348] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: NON_VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: true } }
I20260807 15:10:02.591621 25761 raft_consensus.cc:729] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 582deee640a1481cbf7a09c88c6269f7, State: Initialized, Role: LEARNER
I20260807 15:10:02.591784 25761 consensus_queue.cc:260] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: NON_VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: true } }
I20260807 15:10:02.592263 25767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: NON_VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: true } } }
I20260807 15:10:02.592451 25767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [sys.catalog]: This master's current role is: LEARNER
I20260807 15:10:02.592463 25761 sys_catalog.cc:564] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:02.597426 25780 catalog_manager.cc:1259] Loaded cluster ID: 3115c820c8e740de80daae454f6c5f1c
I20260807 15:10:02.597528 25780 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7: loading cluster ID for follower catalog manager: success
I20260807 15:10:02.600445 25780 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7: acquiring CA information for follower catalog manager: success
I20260807 15:10:02.601953 25780 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 15:10:02.624094 25658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:02.624477 25658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:02.624568 25658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:02.629347 25658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:02.629463 25658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:02.629501 25658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:02.629563 25658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:02.629622 25658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:02.636202 25658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:43293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:44957,127.15.226.125:45651,127.15.226.124:46831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.124.keytab
--principal=oryx/127.15.226.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.124:46831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.226.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:02.637562 25658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:02.638832 25658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:02.649986 25786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:02.650099 25791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:02.650028 25787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:02.650497 25658 server_base.cc:1034] running on GCE node
I20260807 15:10:02.651193 25658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:02.652529 25658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:02.653803 25658 hybrid_clock.cc:648] HybridClock initialized: now 1786115402653767 us; error 67 us; skew 500 ppm
Aug 07 15:10:02 dist-test-slave-3kk6 krb5kdc[24828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115402, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:02.657388 25658 init.cc:377] Logged in from keytab as oryx/127.15.226.124@KRBTEST.COM (short username oryx)
I20260807 15:10:02.658445 25658 webserver.cc:460] Webserver started at http://127.15.226.124:39421/ using document root <none> and password file <none>
I20260807 15:10:02.659180 25658 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:02.659289 25658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:02.659562 25658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:02.662056 25658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/data/instance:
uuid: "7703a9b88b4a46eaa83a4207fe7bd66e"
format_stamp: "Formatted at 2026-08-07 15:10:02 on dist-test-slave-3kk6"
server_key: "fb9986a64cc1cb9ea19c67b381c484f7"
server_key_iv: "4eeb60aaed05622e30a2303846fc02a7"
server_key_version: "encryptionkey@0"
I20260807 15:10:02.662818 25658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/wal/instance:
uuid: "7703a9b88b4a46eaa83a4207fe7bd66e"
format_stamp: "Formatted at 2026-08-07 15:10:02 on dist-test-slave-3kk6"
server_key: "fb9986a64cc1cb9ea19c67b381c484f7"
server_key_iv: "4eeb60aaed05622e30a2303846fc02a7"
server_key_version: "encryptionkey@0"
I20260807 15:10:02.667296 25658 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:10:02.670226 25799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:02.671903 25658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:10:02.672088 25658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/wal
uuid: "7703a9b88b4a46eaa83a4207fe7bd66e"
format_stamp: "Formatted at 2026-08-07 15:10:02 on dist-test-slave-3kk6"
server_key: "fb9986a64cc1cb9ea19c67b381c484f7"
server_key_iv: "4eeb60aaed05622e30a2303846fc02a7"
server_key_version: "encryptionkey@0"
I20260807 15:10:02.672236 25658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:02.688344 25658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:02.692214 25658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:02.692459 25658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:02.706517 25658 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.124:46831
I20260807 15:10:02.706939 25879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.124:46831 every 8 connection(s)
I20260807 15:10:02.707849 25658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/data/info.pb
I20260807 15:10:02.711324 25880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:02.712945 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 25658
I20260807 15:10:02.717337 25880 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:10:02 dist-test-slave-3kk6 krb5kdc[24828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.226.124@KRBTEST.COM
Aug 07 15:10:02 dist-test-slave-3kk6 krb5kdc[24828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115402, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.124@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM
I20260807 15:10:02.730933 25882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.714712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41150 (local address 127.15.226.124:46831)
0807 15:10:02.715469 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:02.715489 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:02.715539 (+ 50us) server_negotiation.cc:407] Connection header received
0807 15:10:02.716723 (+ 1184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:02.716752 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:02.717210 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:02.717643 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:02.718537 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.719863 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.720995 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.721495 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.721676 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.724903 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:02.724909 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:02.724920 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:02.724948 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:02.727136 (+ 2188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.727817 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.727826 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.727835 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.728019 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.728388 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.728394 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.728398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.728885 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:02.729175 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:02.729512 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 15:10:02.729961 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"spinlock_wait_cycles":21248,"thread_start_us":188,"threads_started":1}
I20260807 15:10:02.731262 25885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.719661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.0.0.1:43842)
0807 15:10:02.720286 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:02.720340 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:02.720470 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:02.720799 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:02.720804 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:02.721248 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:02.721484 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.721490 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.722438 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.722442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.723358 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.723369 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.724029 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.724032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.724152 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.724945 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:02.724957 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:02.726881 (+ 1924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:02.729018 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.729029 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.729036 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.729329 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.729650 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.729654 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.729656 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.729745 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.730319 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:02.730328 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:02.730462 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 15:10:02.730617 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 15:10:02.730851 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":268,"threads_started":1}
I20260807 15:10:02.731297 24941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.719801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43842 (local address 127.15.226.126:44957)
0807 15:10:02.719996 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:02.720002 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:02.720447 (+ 445us) server_negotiation.cc:407] Connection header received
0807 15:10:02.720570 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:02.720574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:02.720628 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:02.720728 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:02.721629 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.722157 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.723475 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.723862 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.723984 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.727081 (+ 3097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:02.727087 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:02.727092 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:02.727123 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:02.728885 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.729446 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.729449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.729451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.729548 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.729853 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.729856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.729858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.730166 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:02.730306 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:02.730725 (+ 419us) server_negotiation.cc:299] Negotiation successful
0807 15:10:02.730903 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 15:10:02.732579 25880 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:10:02 dist-test-slave-3kk6 krb5kdc[24828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115402, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.124@KRBTEST.COM for oryx/127.15.226.125@KRBTEST.COM
I20260807 15:10:02.743680 25885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.733185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:45651 (local address 127.0.0.1:35070)
0807 15:10:02.733807 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:02.733820 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:02.733897 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:02.734121 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:02.734123 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:02.734275 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:02.734912 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.734922 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.735924 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.735935 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.736492 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.736501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.737052 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.737055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.737186 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.737744 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:02.737756 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:02.739589 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:02.741776 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.741778 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.741780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.742025 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.742402 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.742404 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.742405 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.742488 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.743011 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:02.743013 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:02.743187 (+ 174us) client_negotiation.cc:769] Sending connection context
0807 15:10:02.743296 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 15:10:02.743483 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556}
I20260807 15:10:02.743911 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.733317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35070 (local address 127.15.226.125:45651)
0807 15:10:02.733715 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:02.733720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:02.733883 (+ 163us) server_negotiation.cc:407] Connection header received
0807 15:10:02.733964 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:02.733967 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:02.734011 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:02.734102 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:02.735091 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.735799 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.736688 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.736939 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.737039 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.739763 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:02.739766 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:02.739768 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:02.739798 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:02.741633 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.742186 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.742190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.742191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.742297 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.742581 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.742583 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.742585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.742915 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:02.743004 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:02.743399 (+ 395us) server_negotiation.cc:299] Negotiation successful
0807 15:10:02.743591 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":183,"threads_started":1}
I20260807 15:10:02.744966 25880 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:10:02 dist-test-slave-3kk6 krb5kdc[24828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115402, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.124@KRBTEST.COM for oryx/127.15.226.124@KRBTEST.COM
I20260807 15:10:02.754949 25882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.745760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41160 (local address 127.15.226.124:46831)
0807 15:10:02.745902 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:02.745908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:02.746014 (+ 106us) server_negotiation.cc:407] Connection header received
0807 15:10:02.746061 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:02.746063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:02.746101 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:02.746171 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:02.746559 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.747236 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.748111 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.748391 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.748570 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.750699 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:02.750705 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:02.750713 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:02.750735 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:02.752711 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.753503 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.753505 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.753507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.753587 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.753876 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.753881 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.753883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.754096 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:02.754240 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:02.754530 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 15:10:02.754812 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":149,"server-negotiator.queue_time_us":60,"spinlock_wait_cycles":7936}
I20260807 15:10:02.754914 25885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.745776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:46831 (local address 127.0.0.1:41160)
0807 15:10:02.745909 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:02.745923 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:02.746006 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:02.746201 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:02.746203 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:02.746362 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:02.746455 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.746460 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.747424 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.747426 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.747970 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.747978 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.748528 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.748531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.748597 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.749137 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:02.749149 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:02.750514 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:02.752889 (+ 2375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.752892 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.752893 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.753300 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.753694 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.753696 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.753698 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.753783 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.754264 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:02.754267 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:02.754358 (+ 91us) client_negotiation.cc:769] Sending connection context
0807 15:10:02.754463 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 15:10:02.754746 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 15:10:02.759115 25880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e: Bootstrap starting.
I20260807 15:10:02.761695 25880 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:02.762570 25880 log.cc:824] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:02.765304 25880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e: No bootstrap required, opened a new log
I20260807 15:10:02.770933 25880 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } }
I20260807 15:10:02.771219 25880 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:02.771257 25880 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7703a9b88b4a46eaa83a4207fe7bd66e, State: Initialized, Role: FOLLOWER
I20260807 15:10:02.771749 25880 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } }
I20260807 15:10:02.772428 25891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } } }
I20260807 15:10:02.772608 25891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:10:02.773125 25880 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [sys.catalog]: configured and running, proceeding with master startup.
W20260807 15:10:02.780305 25906 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:10:02.780439 25906 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:10:02.783922 25658 master_runner.cc:428] Detected that this master 7703a9b88b4a46eaa83a4207fe7bd66e is joining an existing cluster
I20260807 15:10:02.784003 25658 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.226.124:46831
I20260807 15:10:02.785147 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 24973
I20260807 15:10:02.815922 25891 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:10:02.816171 25891 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } }
I20260807 15:10:02.817396 25891 leader_election.cc:288] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1be44596da5d4958b056b014ab975a39 (127.15.226.126:44957), 582deee640a1481cbf7a09c88c6269f7 (127.15.226.125:45651)
I20260807 15:10:02.818432 25724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "582deee640a1481cbf7a09c88c6269f7" is_pre_election: true
I20260807 15:10:02.818691 25724 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 1 LEARNER]: Handling vote request from an unknown peer 7703a9b88b4a46eaa83a4207fe7bd66e
I20260807 15:10:02.818859 25724 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 7703a9b88b4a46eaa83a4207fe7bd66e for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260807 15:10:02.819569 24905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1be44596da5d4958b056b014ab975a39" is_pre_election: true
I20260807 15:10:02.820430 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:37605
--local_ip_for_outbound_sockets=127.15.226.65
--tserver_master_addrs=127.15.226.126:44957
--webserver_port=45699
--webserver_interface=127.15.226.65
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.65.keytab
--principal=oryx/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:43293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.226.126:44957,127.15.226.125:45651
--tserver_master_addrs=127.15.226.126:44957,127.15.226.125:45651,127.15.226.124:46831 with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:10:02.820863 24941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.787889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43854 (local address 127.15.226.126:44957)
0807 15:10:02.788101 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:02.788108 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:02.802067 (+ 13959us) server_negotiation.cc:407] Connection header received
0807 15:10:02.802399 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:02.802406 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:02.802492 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:02.802576 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:02.804536 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.805778 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.808296 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.808830 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.808983 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.815803 (+ 6820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:02.815808 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:02.815811 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:02.815848 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:02.818738 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.819301 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.819306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.819308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.819413 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.819773 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.819777 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.819779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.820032 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:02.820120 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:02.820546 (+ 426us) server_negotiation.cc:299] Negotiation successful
0807 15:10:02.820691 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 15:10:02.821439 25918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.787549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.0.0.1:43854)
0807 15:10:02.801903 (+ 14354us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:02.801929 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:02.802208 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:02.802877 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:02.802883 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:02.803372 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:02.803652 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.803670 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.806074 (+ 2404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.806081 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.808055 (+ 1974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.808075 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.809321 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.809329 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.809558 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.814875 (+ 5317us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:02.814893 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:02.815579 (+ 686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:02.818902 (+ 3323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.818906 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.818908 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.819180 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.819567 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.819570 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.819572 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.819662 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.820181 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:02.820184 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:02.820286 (+ 102us) client_negotiation.cc:769] Sending connection context
0807 15:10:02.821071 (+ 785us) client_negotiation.cc:240] Negotiation successful
0807 15:10:02.821243 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14095,"thread_start_us":173,"threads_started":1}
I20260807 15:10:02.822098 25328 heartbeater.cc:503] Master 127.15.226.126:44957 was elected leader, sending a full tablet report...
I20260807 15:10:02.822533 25802 leader_election.cc:302] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7703a9b88b4a46eaa83a4207fe7bd66e; no voters: 1be44596da5d4958b056b014ab975a39, 582deee640a1481cbf7a09c88c6269f7
I20260807 15:10:02.823136 25891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:02.824651 25891 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 15:10:02.832881 25918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.800779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:46831 (local address 127.0.0.1:41172)
0807 15:10:02.823781 (+ 23002us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:02.823798 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:02.823941 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:02.824353 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:02.824356 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:02.824557 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:02.824672 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.824678 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.825687 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.825690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.826572 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.826584 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.827238 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.827242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.827385 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.828056 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:02.828071 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:02.828538 (+ 467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:02.830854 (+ 2316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.830860 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.830864 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.831182 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.831541 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.831545 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.831547 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.831652 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.832211 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:02.832215 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:02.832319 (+ 104us) client_negotiation.cc:769] Sending connection context
0807 15:10:02.832454 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 15:10:02.832625 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22874,"mutex_wait_us":31,"thread_start_us":304,"threads_started":1}
I20260807 15:10:02.833361 25882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.800273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41172 (local address 127.15.226.124:46831)
0807 15:10:02.800476 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:02.800485 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:02.823859 (+ 23374us) server_negotiation.cc:407] Connection header received
0807 15:10:02.824073 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:02.824076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:02.824142 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:02.824190 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:02.824786 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.825566 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.826724 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.827115 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.827267 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.828685 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:02.828688 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:02.828691 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:02.828712 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:02.830714 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.831320 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.831326 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.831329 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.831427 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.831785 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.831790 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.831793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.832094 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:02.832219 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:02.832548 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 15:10:02.832699 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":77}
I20260807 15:10:02.857331 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.814824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35082 (local address 127.15.226.125:45651)
0807 15:10:02.815010 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:02.815017 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:02.815039 (+ 22us) server_negotiation.cc:407] Connection header received
0807 15:10:02.815098 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:02.815104 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:02.815166 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:02.815288 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:02.825511 (+ 10223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.826370 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.828327 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.828777 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.828913 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.852666 (+ 23753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:02.852674 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:02.852676 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:02.852712 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:02.855024 (+ 2312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.855847 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.855851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.855853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.855951 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.856301 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.856305 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.856307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.856541 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:02.856628 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:02.856989 (+ 361us) server_negotiation.cc:299] Negotiation successful
0807 15:10:02.857143 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 15:10:02.857954 25917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.789516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:45651 (local address 127.0.0.1:35082)
0807 15:10:02.804108 (+ 14592us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:02.804132 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:02.804349 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:02.824847 (+ 20498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:02.824853 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:02.825123 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:02.825272 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.825283 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.826590 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.826596 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.828170 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.828185 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.829120 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.829124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.829250 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.851592 (+ 22342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:02.851613 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:02.852397 (+ 784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:02.855176 (+ 2779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.855180 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.855182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.855487 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.856068 (+ 581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.856071 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.856073 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.856170 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.856690 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:02.856693 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:02.856799 (+ 106us) client_negotiation.cc:769] Sending connection context
0807 15:10:02.857574 (+ 775us) client_negotiation.cc:240] Negotiation successful
0807 15:10:02.857769 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14457,"mutex_wait_us":3,"thread_start_us":1182,"threads_started":1}
Aug 07 15:10:02 dist-test-slave-3kk6 krb5kdc[24828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115401, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.126@KRBTEST.COM for oryx/127.15.226.124@KRBTEST.COM
I20260807 15:10:02.864583 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.853511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:46831 (local address 127.0.0.1:41182)
0807 15:10:02.853733 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:02.853748 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:02.853836 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:02.854478 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:02.854482 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:02.854743 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:02.854890 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.854899 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.856094 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.856098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.856948 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.856960 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.857879 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.857883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.857990 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.858842 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:02.858858 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:02.860615 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:02.862741 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.862746 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.862748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.863027 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.863344 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.863347 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.863349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.863455 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.863931 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:02.863934 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:02.864062 (+ 128us) client_negotiation.cc:769] Sending connection context
0807 15:10:02.864181 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 15:10:02.864395 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141}
I20260807 15:10:02.864581 25882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.854001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41182 (local address 127.15.226.124:46831)
0807 15:10:02.854161 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:02.854165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:02.854179 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:10:02.854266 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:02.854269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:02.854329 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:02.855038 (+ 709us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:02.855159 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.855953 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.857104 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.857734 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.859220 (+ 1486us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.860784 (+ 1564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:02.860789 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:02.860792 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:02.860820 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:02.862559 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.863140 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.863144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.863146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.863235 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.863565 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.863568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.863570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.863822 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:02.863922 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:02.864272 (+ 350us) server_negotiation.cc:299] Negotiation successful
0807 15:10:02.864416 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 15:10:02.866309 24892 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.226.124:46831: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e"
member_type: NON_VOTER
last_known_addr {
host: "127.15.226.124"
port: 46831
}
attrs {
promote: true
}
}
dest_uuid: "1be44596da5d4958b056b014ab975a39"
cas_config_opid_index: 5
I20260807 15:10:02.867056 24892 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: NON_VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: true } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: NON_VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: true } }
I20260807 15:10:02.868239 25653 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 7703a9b88b4a46eaa83a4207fe7bd66e (127.15.226.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: NON_VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: true } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: NON_VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: true } } }
I20260807 15:10:02.869650 25839 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 1 FOLLOWER]: Refusing update from remote peer 1be44596da5d4958b056b014ab975a39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 15:10:02.869969 25653 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.226.124:46831
I20260807 15:10:02.870180 25653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1be44596da5d4958b056b014ab975a39" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: NON_VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: true } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: NON_VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: true } } }
I20260807 15:10:02.870272 25650 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: NON_VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:02.871090 25658 master.cc:537] Master@127.15.226.124:46831 shutting down...
I20260807 15:10:02.870281 25653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [sys.catalog]: This master's current role is: LEADER
W20260807 15:10:02.872824 24869 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 -> Peer 7703a9b88b4a46eaa83a4207fe7bd66e (127.15.226.124:46831): Couldn't send request to peer 7703a9b88b4a46eaa83a4207fe7bd66e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 15:10:02.877494 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.868767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35094 (local address 127.15.226.125:45651)
0807 15:10:02.871959 (+ 3192us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:02.871964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:02.872282 (+ 318us) server_negotiation.cc:407] Connection header received
0807 15:10:02.872353 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:02.872356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:02.872421 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:02.872509 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:02.872901 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.874016 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.875631 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.876309 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.876449 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.876619 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:02.877210 (+ 591us) server_negotiation.cc:299] Negotiation successful
0807 15:10:02.877300 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3072}
I20260807 15:10:02.877903 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.868484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:45651 (local address 127.0.0.1:35094)
0807 15:10:02.872075 (+ 3591us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:02.872092 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:02.872193 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:02.872651 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:02.872654 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:02.872671 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:02.872774 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.872780 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.874161 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.874165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.875471 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.875485 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.876698 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.876701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.876803 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.876859 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 15:10:02.876939 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 15:10:02.877028 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3501}
I20260807 15:10:02.879001 25724 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 1 LEARNER]: Refusing update from remote peer 1be44596da5d4958b056b014ab975a39: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 15:10:02.879549 25653 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: NON_VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 15:10:02.881667 25724 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 7703a9b88b4a46eaa83a4207fe7bd66e (127.15.226.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: NON_VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: true } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: NON_VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: true } } }
I20260807 15:10:02.886240 25658 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:10:02.886730 25658 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:02.886847 25658 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e: stopping tablet replica
I20260807 15:10:02.886893 25767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1be44596da5d4958b056b014ab975a39. Latest consensus state: current_term: 1 leader_uuid: "1be44596da5d4958b056b014ab975a39" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: NON_VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: true } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: NON_VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: true } } }
I20260807 15:10:02.887044 25767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [sys.catalog]: This master's current role is: LEARNER
I20260807 15:10:02.887151 25767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1be44596da5d4958b056b014ab975a39" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: NON_VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: true } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: NON_VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: true } } }
I20260807 15:10:02.887217 25767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [sys.catalog]: This master's current role is: LEARNER
I20260807 15:10:02.887863 25653 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39: attempting to promote NON_VOTER 582deee640a1481cbf7a09c88c6269f7 to VOTER
I20260807 15:10:02.888437 25653 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: NON_VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: true } }
I20260807 15:10:02.893249 25658 master.cc:552] Master@127.15.226.124:46831 shutdown complete.
I20260807 15:10:02.893404 25658 master_runner.cc:305] Clearing existing system tablet
I20260807 15:10:02.893448 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.893070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:46831 (local address 127.0.0.1:41196)
0807 15:10:02.893261 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:02.893351 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:46831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260807 15:10:02.893815 25658 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:02.894017 25658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:02.894333 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.894063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:46831 (local address 127.0.0.1:41206)
0807 15:10:02.894214 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:02.894271 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:46831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260807 15:10:02.894484 24869 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 -> Peer 7703a9b88b4a46eaa83a4207fe7bd66e (127.15.226.124:46831): Couldn't send request to peer 7703a9b88b4a46eaa83a4207fe7bd66e. Status: Network error: Client connection negotiation failed: client connection to 127.15.226.124:46831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 15:10:02.895382 25722 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 1 LEARNER]: Refusing update from remote peer 1be44596da5d4958b056b014ab975a39: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 15:10:02.895848 25653 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 15:10:02.896289 25658 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:10:02.897919 25654 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 582deee640a1481cbf7a09c88c6269f7 (127.15.226.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: NON_VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: true } } }
I20260807 15:10:02.898322 25722 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 582deee640a1481cbf7a09c88c6269f7 (127.15.226.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: NON_VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: true } } }
I20260807 15:10:02.899871 25654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1be44596da5d4958b056b014ab975a39" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: NON_VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: true } } }
I20260807 15:10:02.899995 25654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:02.900123 25654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1be44596da5d4958b056b014ab975a39" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: NON_VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: true } } }
I20260807 15:10:02.900187 25654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:02.901335 25922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1be44596da5d4958b056b014ab975a39" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: NON_VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: true } } }
I20260807 15:10:02.901515 25922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:10:02.902882 25928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:02.903146 25658 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:02.903316 25658 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260807 15:10:02.903363 25658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/wal
uuid: "7703a9b88b4a46eaa83a4207fe7bd66e"
format_stamp: "Formatted at 2026-08-07 15:10:02 on dist-test-slave-3kk6"
server_key: "fb9986a64cc1cb9ea19c67b381c484f7"
server_key_iv: "4eeb60aaed05622e30a2303846fc02a7"
server_key_version: "encryptionkey@0"
I20260807 15:10:02.905675 25658 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:02.908404 25658 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 15:10:02.908517 25658 log.cc:1196] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 15:10:02.908823 25658 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e: Deleting consensus metadata
I20260807 15:10:02.909003 25658 master_runner.cc:315] Copying system tablet from 127.15.226.126:44957
I20260807 15:10:02.910301 25658 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e: tablet copy: Beginning tablet copy session from remote peer at address 127.15.226.126:44957
I20260807 15:10:02.920850 25937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.911185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.0.0.1:43860)
0807 15:10:02.911679 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:02.911710 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:02.911861 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:02.912276 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:02.912279 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:02.912534 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:02.912654 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.912662 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.913801 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.913805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.914740 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:02.914752 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.915419 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.915424 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:02.915512 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.916196 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:02.916211 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:02.916707 (+ 496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:02.918989 (+ 2282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.918993 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.918995 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.919310 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.919656 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:02.919659 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:02.919661 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:02.919772 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:02.920239 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:02.920243 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:02.920369 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 15:10:02.920489 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 15:10:02.920661 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":125,"threads_started":1}
I20260807 15:10:02.920869 24941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:02.911371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43860 (local address 127.15.226.126:44957)
0807 15:10:02.911524 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:02.911528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:02.911840 (+ 312us) server_negotiation.cc:407] Connection header received
0807 15:10:02.911961 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:02.911965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:02.912025 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:02.912125 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:02.912818 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.913651 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.914899 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:02.915268 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:02.915389 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:02.916885 (+ 1496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:02.916890 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:02.916893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:02.916924 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:02.918855 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.919440 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.919444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.919446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.919548 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:02.919878 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:02.919881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:02.919882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:02.920127 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:02.920230 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:02.920568 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 15:10:02.920707 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 15:10:02.921505 24917 tablet_copy_service.cc:140] P 1be44596da5d4958b056b014ab975a39: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 7703a9b88b4a46eaa83a4207fe7bd66e ({username='oryx', principal='oryx/127.15.226.124@KRBTEST.COM'} at 127.0.0.1:43860)
I20260807 15:10:02.921624 24917 tablet_copy_service.cc:161] P 1be44596da5d4958b056b014ab975a39: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 7703a9b88b4a46eaa83a4207fe7bd66e at {username='oryx', principal='oryx/127.15.226.124@KRBTEST.COM'} at 127.0.0.1:43860: session id = 7703a9b88b4a46eaa83a4207fe7bd66e-00000000000000000000000000000000
I20260807 15:10:02.922866 24917 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 15:10:02.924494 25658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 15:10:02.927639 25658 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e: tablet copy: Starting download of 0 data blocks...
I20260807 15:10:02.927881 25658 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e: tablet copy: Starting download of 1 WAL segments...
I20260807 15:10:02.930080 25658 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 15:10:02.932507 24917 tablet_copy_service.cc:342] P 1be44596da5d4958b056b014ab975a39: Request end of tablet copy session 7703a9b88b4a46eaa83a4207fe7bd66e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.226.124@KRBTEST.COM'} at 127.0.0.1:43860
I20260807 15:10:02.932622 24917 tablet_copy_service.cc:433] P 1be44596da5d4958b056b014ab975a39: ending tablet copy session 7703a9b88b4a46eaa83a4207fe7bd66e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 7703a9b88b4a46eaa83a4207fe7bd66e
I20260807 15:10:02.940783 25658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:02.942960 25938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:02.942981 25941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:02.943837 25658 server_base.cc:1034] running on GCE node
W20260807 15:10:02.946933 25939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:02.947240 25658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:02.947659 25658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:02.948830 25658 hybrid_clock.cc:648] HybridClock initialized: now 1786115402948807 us; error 53 us; skew 500 ppm
Aug 07 15:10:02 dist-test-slave-3kk6 krb5kdc[24828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115402, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:02.950820 25658 init.cc:377] Logged in from keytab as oryx/127.15.226.124@KRBTEST.COM (short username oryx)
I20260807 15:10:02.951399 25658 webserver.cc:460] Webserver started at http://127.15.226.124:41123/ using document root <none> and password file <none>
I20260807 15:10:02.951609 25658 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:02.951658 25658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:02.953203 25658 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 15:10:02.954417 25952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:02.954741 25658 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 15:10:02.954833 25658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/wal
uuid: "7703a9b88b4a46eaa83a4207fe7bd66e"
format_stamp: "Formatted at 2026-08-07 15:10:02 on dist-test-slave-3kk6"
server_key: "fb9986a64cc1cb9ea19c67b381c484f7"
server_key_iv: "4eeb60aaed05622e30a2303846fc02a7"
server_key_version: "encryptionkey@0"
I20260807 15:10:02.954978 25658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:02.963587 25658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:02.966095 25658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:02.972757 25658 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.124:46831
I20260807 15:10:02.972793 26019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.124:46831 every 8 connection(s)
I20260807 15:10:02.973578 25658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/master-2/data/info.pb
I20260807 15:10:02.975627 26020 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:10:02.976821 26020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e: Bootstrap starting.
I20260807 15:10:02.990417 26020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:10:02.990823 26020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e: Bootstrap complete.
I20260807 15:10:02.991619 26020 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: NON_VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: true } }
I20260807 15:10:02.991794 26020 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7703a9b88b4a46eaa83a4207fe7bd66e, State: Initialized, Role: LEARNER
I20260807 15:10:02.991909 26020 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: NON_VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: true } }
I20260807 15:10:02.992277 26030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: NON_VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: true } } }
I20260807 15:10:02.992421 26030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [sys.catalog]: This master's current role is: LEARNER
I20260807 15:10:02.992436 26020 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [sys.catalog]: configured and running, proceeding with master startup.
W20260807 15:10:02.995678 25919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:02.995991 25919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:02.996075 25919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 15:10:02.996279 26045 catalog_manager.cc:1259] Loaded cluster ID: 3115c820c8e740de80daae454f6c5f1c
I20260807 15:10:02.996351 26045 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e: loading cluster ID for follower catalog manager: success
I20260807 15:10:02.997933 26045 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e: acquiring CA information for follower catalog manager: success
I20260807 15:10:02.999274 26045 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 15:10:03.000706 25919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:03.000773 25919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:03.000888 25919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:10:03.006901 25919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:43293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.65.keytab
--principal=oryx/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:37605
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=45699
--tserver_master_addrs=127.15.226.126:44957,127.15.226.125:45651,127.15.226.124:46831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:03.008136 25919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:03.009243 25919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:03.020627 26051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:03.020783 25919 server_base.cc:1034] running on GCE node
W20260807 15:10:03.020627 26053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:03.020674 26049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:03.021492 25919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:03.022254 25919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:03.023501 25919 hybrid_clock.cc:648] HybridClock initialized: now 1786115403023485 us; error 32 us; skew 500 ppm
Aug 07 15:10:03 dist-test-slave-3kk6 krb5kdc[24828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115403, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:03.027362 25919 init.cc:377] Logged in from keytab as oryx/127.15.226.65@KRBTEST.COM (short username oryx)
I20260807 15:10:03.028462 25919 webserver.cc:460] Webserver started at http://127.15.226.65:45699/ using document root <none> and password file <none>
I20260807 15:10:03.029135 25919 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:03.029276 25919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:03.034278 25919 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:10:03.037726 26062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:03.039491 25919 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260807 15:10:03.039638 25919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "60340eaffc12444aaf7a470a0634f9ac"
format_stamp: "Formatted at 2026-08-07 15:10:01 on dist-test-slave-3kk6"
server_key: "d0573ff99f213938b9f0fb9ee7ccfdcd"
server_key_iv: "f65caad6614b6b6c1aaa01c769925ca2"
server_key_version: "encryptionkey@0"
I20260807 15:10:03.040266 25919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:03.060439 25513 heartbeater.cc:503] Master 127.15.226.126:44957 was elected leader, sending a full tablet report...
I20260807 15:10:03.064224 25919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:03.068706 25919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:03.068925 25919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:03.069597 25919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:03.070616 25919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:03.070729 25919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:03.070787 25919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:03.070804 25919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:03.084620 25919 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:37605
I20260807 15:10:03.084641 26227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:37605 every 8 connection(s)
I20260807 15:10:03.085812 25919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:10:03.094610 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 25919
I20260807 15:10:03.094966 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 25150
Aug 07 15:10:03 dist-test-slave-3kk6 krb5kdc[24828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115403, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.125@KRBTEST.COM
Aug 07 15:10:03 dist-test-slave-3kk6 krb5kdc[24828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115403, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM
I20260807 15:10:03.109447 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.088025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:37733 (local address 127.15.226.125:45651)
0807 15:10:03.088221 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:03.088227 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:03.089797 (+ 1570us) server_negotiation.cc:407] Connection header received
0807 15:10:03.091258 (+ 1461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:03.091261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:03.091328 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:03.091390 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:03.093218 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.093986 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.096973 (+ 2987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.097331 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.097421 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.103552 (+ 6131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:03.103560 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:03.103566 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:03.103611 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:03.105719 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.106914 (+ 1195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.106922 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.106927 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.107061 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.107680 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.107686 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.107697 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.107982 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:03.108090 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:03.109062 (+ 972us) server_negotiation.cc:299] Negotiation successful
0807 15:10:03.109227 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 15:10:03.111294 26236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.088509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:45651 (local address 127.15.226.65:37733)
0807 15:10:03.089679 (+ 1170us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:03.089694 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:03.090943 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:03.091656 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:03.091659 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:03.092319 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:03.093019 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.093068 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.094958 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.094962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.096758 (+ 1796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.096772 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.097682 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.097687 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.097994 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.099255 (+ 1261us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:03.099286 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:03.102616 (+ 3330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:03.106031 (+ 3415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:03.106044 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:03.106075 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:03.106720 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:03.107215 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:03.107222 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:03.107227 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:03.107525 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:03.108131 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:03.108143 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:03.108667 (+ 524us) client_negotiation.cc:769] Sending connection context
0807 15:10:03.108995 (+ 328us) client_negotiation.cc:240] Negotiation successful
0807 15:10:03.109540 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1071,"mutex_wait_us":98,"spinlock_wait_cycles":62080,"thread_start_us":205,"threads_started":1}
I20260807 15:10:03.117496 26231 heartbeater.cc:347] Connected to a master server at 127.15.226.125:45651
I20260807 15:10:03.117851 26231 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:03.118577 26231 heartbeater.cc:511] Master 127.15.226.125:45651 requested a full tablet report, sending...
I20260807 15:10:03.121061 25710 ts_manager.cc:194] Registered new tserver with Master: 60340eaffc12444aaf7a470a0634f9ac (127.15.226.65:37605)
Aug 07 15:10:03 dist-test-slave-3kk6 krb5kdc[24828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115403, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.124@KRBTEST.COM
I20260807 15:10:03.121731 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.088743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.65:53697)
0807 15:10:03.090282 (+ 1539us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:03.090296 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:03.090999 (+ 703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:03.091604 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:03.091612 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:03.092234 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:03.093018 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.093053 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.095362 (+ 2309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.095365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.096961 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.096972 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.097923 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.097927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.098070 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.099333 (+ 1263us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:03.099360 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:03.103098 (+ 3738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:03.118880 (+ 15782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:03.118887 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:03.118893 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:03.119221 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:03.119652 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:03.119658 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:03.119663 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:03.119768 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:03.120307 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:03.120313 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:03.120441 (+ 128us) client_negotiation.cc:769] Sending connection context
0807 15:10:03.120551 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 15:10:03.120739 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1475,"mutex_wait_us":94,"spinlock_wait_cycles":7552,"thread_start_us":114,"threads_started":1}
I20260807 15:10:03.122466 24941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.088903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:53697 (local address 127.15.226.126:44957)
0807 15:10:03.089030 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:03.089034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:03.090384 (+ 1350us) server_negotiation.cc:407] Connection header received
0807 15:10:03.091236 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:03.091240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:03.091306 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:03.091394 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:03.094256 (+ 2862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.095150 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.097182 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.097625 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.097719 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.109152 (+ 11433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:03.109157 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:03.109161 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:03.109211 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:03.118712 (+ 9501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.119391 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.119397 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.119402 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.119527 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.119879 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.119885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.119890 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.120156 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:03.120257 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:03.122041 (+ 1784us) server_negotiation.cc:299] Negotiation successful
0807 15:10:03.122205 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 15:10:03.123121 26230 heartbeater.cc:347] Connected to a master server at 127.15.226.126:44957
I20260807 15:10:03.123221 26230 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:03.123479 26230 heartbeater.cc:511] Master 127.15.226.126:44957 requested a full tablet report, sending...
I20260807 15:10:03.124524 24892 ts_manager.cc:194] Re-registered known tserver with Master: 60340eaffc12444aaf7a470a0634f9ac (127.15.226.65:37605)
I20260807 15:10:03.124913 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:44821
--local_ip_for_outbound_sockets=127.15.226.66
--tserver_master_addrs=127.15.226.126:44957
--webserver_port=34537
--webserver_interface=127.15.226.66
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.66.keytab
--principal=oryx/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:43293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.226.126:44957,127.15.226.125:45651
--tserver_master_addrs=127.15.226.126:44957,127.15.226.125:45651,127.15.226.124:46831 with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:10:03.125382 24892 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.226.65@KRBTEST.COM'} at 127.15.226.65:53697
I20260807 15:10:03.126310 26233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.088049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:46033 (local address 127.15.226.124:46831)
0807 15:10:03.088436 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:03.088441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:03.089734 (+ 1293us) server_negotiation.cc:407] Connection header received
0807 15:10:03.091270 (+ 1536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:03.091273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:03.091328 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:03.091390 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:03.093888 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.095096 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.118053 (+ 22957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.118457 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.118558 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.121829 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:03.121834 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:03.121837 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:03.121871 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:03.124095 (+ 2224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.124663 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.124667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.124669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.124772 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.125162 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.125168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.125173 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.125470 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:03.125560 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:03.125992 (+ 432us) server_negotiation.cc:299] Negotiation successful
0807 15:10:03.126151 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":89,"threads_started":1}
I20260807 15:10:03.126369 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.088507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:46831 (local address 127.15.226.65:46033)
0807 15:10:03.089532 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:03.089603 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:03.090943 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:03.091624 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:03.091628 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:03.093640 (+ 2012us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:03.093745 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.093750 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.116866 (+ 23116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.116872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.117834 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.117851 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.118987 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.118994 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.119163 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.119861 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:03.119879 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:03.121620 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:03.124202 (+ 2582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:03.124205 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:03.124207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:03.124509 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:03.124909 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:03.124912 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:03.124914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:03.125023 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:03.125642 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:03.125647 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:03.125765 (+ 118us) client_negotiation.cc:769] Sending connection context
0807 15:10:03.125875 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 15:10:03.126062 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"spinlock_wait_cycles":2176,"thread_start_us":200,"threads_started":1}
I20260807 15:10:03.127692 26228 heartbeater.cc:347] Connected to a master server at 127.15.226.124:46831
I20260807 15:10:03.127815 26228 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:03.128021 26228 heartbeater.cc:511] Master 127.15.226.124:46831 requested a full tablet report, sending...
I20260807 15:10:03.129240 25974 ts_manager.cc:194] Registered new tserver with Master: 60340eaffc12444aaf7a470a0634f9ac (127.15.226.65:37605)
W20260807 15:10:03.253372 26240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:03.253772 26240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:03.253893 26240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:03.258534 26240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:03.258653 26240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:03.258787 26240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:10:03.264744 26240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:43293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.66.keytab
--principal=oryx/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:44821
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=34537
--tserver_master_addrs=127.15.226.126:44957,127.15.226.125:45651,127.15.226.124:46831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:03.266104 26240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:03.267372 26240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:03.278525 26248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:03.278630 26246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:03.278517 26245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:03.279039 26240 server_base.cc:1034] running on GCE node
I20260807 15:10:03.279533 26240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:03.280210 26240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:03.281421 26240 hybrid_clock.cc:648] HybridClock initialized: now 1786115403281389 us; error 49 us; skew 500 ppm
Aug 07 15:10:03 dist-test-slave-3kk6 krb5kdc[24828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115403, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:03.284548 26240 init.cc:377] Logged in from keytab as oryx/127.15.226.66@KRBTEST.COM (short username oryx)
I20260807 15:10:03.285521 26240 webserver.cc:460] Webserver started at http://127.15.226.66:34537/ using document root <none> and password file <none>
I20260807 15:10:03.286098 26240 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:03.286190 26240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:03.290752 26240 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260807 15:10:03.293292 26256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:03.294967 26240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 15:10:03.295146 26240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "f448c8ea9dc5445496f57727ac1061bb"
format_stamp: "Formatted at 2026-08-07 15:10:01 on dist-test-slave-3kk6"
server_key: "12f48013d1d9cff85801c5fdf5842ad5"
server_key_iv: "6ded28c4e262fb5d179925d4fe646381"
server_key_version: "encryptionkey@0"
I20260807 15:10:03.295886 26240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:03.314914 26240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:03.318331 26240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:03.318514 26240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:03.319221 26240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:03.320165 26240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:03.320217 26240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:03.320268 26240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:03.320284 26240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:03.331352 26240 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:44821
I20260807 15:10:03.331378 26422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:44821 every 8 connection(s)
I20260807 15:10:03.332690 26240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:10:03.337715 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 26240
I20260807 15:10:03.337953 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 25334
Aug 07 15:10:03 dist-test-slave-3kk6 krb5kdc[24828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115403, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.125@KRBTEST.COM
Aug 07 15:10:03 dist-test-slave-3kk6 krb5kdc[24828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115403, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM
I20260807 15:10:03.368381 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.335151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:37543 (local address 127.15.226.125:45651)
0807 15:10:03.335623 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:03.335627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:03.337071 (+ 1444us) server_negotiation.cc:407] Connection header received
0807 15:10:03.348262 (+ 11191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:03.348270 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:03.348375 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:03.348459 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:03.352024 (+ 3565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.353351 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.356961 (+ 3610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.357509 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.357652 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.362925 (+ 5273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:03.362933 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:03.362939 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:03.362980 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:03.365071 (+ 2091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.365915 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.365922 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.365927 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.366037 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.366604 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.366611 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.366615 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.366973 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:03.367073 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:03.367957 (+ 884us) server_negotiation.cc:299] Negotiation successful
0807 15:10:03.368124 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366}
I20260807 15:10:03.370630 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:43089
--local_ip_for_outbound_sockets=127.15.226.67
--tserver_master_addrs=127.15.226.126:44957
--webserver_port=46813
--webserver_interface=127.15.226.67
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.67.keytab
--principal=oryx/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:43293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.226.126:44957,127.15.226.125:45651
--tserver_master_addrs=127.15.226.126:44957,127.15.226.125:45651,127.15.226.124:46831 with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:10:03.371053 26428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.335519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:45651 (local address 127.15.226.66:37543)
0807 15:10:03.336964 (+ 1445us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:03.336976 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:03.347710 (+ 10734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:03.349217 (+ 1507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:03.349237 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:03.350329 (+ 1092us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:03.351663 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.351724 (+ 61us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.353763 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.353770 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.356672 (+ 2902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.356698 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.358005 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.358017 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.358397 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.359913 (+ 1516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:03.359950 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:03.362609 (+ 2659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:03.365238 (+ 2629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:03.365250 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:03.365276 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:03.365769 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:03.366166 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:03.366172 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:03.366177 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:03.366461 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:03.367159 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:03.367170 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:03.367591 (+ 421us) client_negotiation.cc:769] Sending connection context
0807 15:10:03.368628 (+ 1037us) client_negotiation.cc:240] Negotiation successful
0807 15:10:03.369119 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1324,"mutex_wait_us":275,"spinlock_wait_cycles":37632,"thread_start_us":160,"threads_started":1}
Aug 07 15:10:03 dist-test-slave-3kk6 krb5kdc[24828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115403, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.124@KRBTEST.COM
I20260807 15:10:03.372994 24941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.335461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:59607 (local address 127.15.226.126:44957)
0807 15:10:03.335689 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:03.335693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:03.336734 (+ 1041us) server_negotiation.cc:407] Connection header received
0807 15:10:03.361460 (+ 24726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:03.361466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:03.361548 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:03.361618 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:03.362543 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.363478 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.364884 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.365216 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.365303 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.368242 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:03.368248 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:03.368254 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:03.368289 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:03.370633 (+ 2344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.371299 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.371307 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.371312 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.371419 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.371795 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.371799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.371800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.372086 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:03.372175 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:03.372614 (+ 439us) server_negotiation.cc:299] Negotiation successful
0807 15:10:03.372781 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260807 15:10:03.373176 26425 heartbeater.cc:347] Connected to a master server at 127.15.226.125:45651
I20260807 15:10:03.373509 26425 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:03.374193 26425 heartbeater.cc:511] Master 127.15.226.125:45651 requested a full tablet report, sending...
I20260807 15:10:03.375669 26427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.335519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.66:59607)
0807 15:10:03.336523 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:03.336592 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:03.361208 (+ 24616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:03.361840 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:03.361843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:03.362216 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:03.362375 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.362383 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.363715 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.363718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.364689 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.364696 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.365497 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.365500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.365602 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.366155 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:03.366175 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:03.368052 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:03.370838 (+ 2786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:03.370842 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:03.370845 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:03.371144 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:03.371552 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:03.371558 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:03.371563 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:03.371663 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:03.372261 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:03.372267 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:03.372407 (+ 140us) client_negotiation.cc:769] Sending connection context
0807 15:10:03.374727 (+ 2320us) client_negotiation.cc:240] Negotiation successful
0807 15:10:03.374921 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"spinlock_wait_cycles":17408,"thread_start_us":812,"threads_started":1}
I20260807 15:10:03.375732 25710 ts_manager.cc:194] Registered new tserver with Master: f448c8ea9dc5445496f57727ac1061bb (127.15.226.66:44821)
I20260807 15:10:03.376502 26424 heartbeater.cc:347] Connected to a master server at 127.15.226.126:44957
I20260807 15:10:03.376602 26424 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:03.376853 26424 heartbeater.cc:511] Master 127.15.226.126:44957 requested a full tablet report, sending...
I20260807 15:10:03.377076 26429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.335519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:46831 (local address 127.15.226.66:42717)
0807 15:10:03.337103 (+ 1584us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:03.337117 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:03.363129 (+ 26012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:03.364273 (+ 1144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:03.364277 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:03.364528 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:03.364674 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.364682 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.366127 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.366131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.367091 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.367104 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.368035 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.368038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.368166 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.370027 (+ 1861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:03.370051 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:03.372464 (+ 2413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:03.374470 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:03.374474 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:03.374476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:03.374784 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:03.375167 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:03.375169 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:03.375171 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:03.375269 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:03.376408 (+ 1139us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:03.376412 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:03.376526 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 15:10:03.376697 (+ 171us) client_negotiation.cc:240] Negotiation successful
0807 15:10:03.376881 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1486,"mutex_wait_us":341,"spinlock_wait_cycles":14464,"thread_start_us":145,"threads_started":1}
I20260807 15:10:03.377071 26233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.335315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:42717 (local address 127.15.226.124:46831)
0807 15:10:03.335803 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:03.335806 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:03.337279 (+ 1473us) server_negotiation.cc:407] Connection header received
0807 15:10:03.363655 (+ 26376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:03.363679 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:03.363763 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:03.363826 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:03.364888 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.365650 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.367255 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.367698 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.367809 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.372665 (+ 4856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:03.372669 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:03.372672 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:03.372720 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:03.374338 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.374925 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.374928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.374930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.375030 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.375400 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.375404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.375405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.375673 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:03.375791 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:03.376784 (+ 993us) server_negotiation.cc:299] Negotiation successful
0807 15:10:03.376929 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396}
I20260807 15:10:03.377629 24892 ts_manager.cc:194] Re-registered known tserver with Master: f448c8ea9dc5445496f57727ac1061bb (127.15.226.66:44821)
I20260807 15:10:03.377703 26423 heartbeater.cc:347] Connected to a master server at 127.15.226.124:46831
I20260807 15:10:03.377794 26423 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:03.377969 26423 heartbeater.cc:511] Master 127.15.226.124:46831 requested a full tablet report, sending...
I20260807 15:10:03.378458 24892 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.226.66@KRBTEST.COM'} at 127.15.226.66:59607
I20260807 15:10:03.378794 25974 ts_manager.cc:194] Registered new tserver with Master: f448c8ea9dc5445496f57727ac1061bb (127.15.226.66:44821)
I20260807 15:10:03.438954 26433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.434329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:46831 (local address 127.0.0.1:41216)
0807 15:10:03.434633 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:03.434705 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:03.434867 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:03.435189 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:03.435191 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:03.435206 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:03.435332 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.435337 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.436419 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.436421 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.437235 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.437246 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.438250 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.438253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.438348 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.438376 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 15:10:03.438540 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 15:10:03.438641 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":84,"threads_started":1}
I20260807 15:10:03.438949 26233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.434520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41216 (local address 127.15.226.124:46831)
0807 15:10:03.434714 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:03.434719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:03.434834 (+ 115us) server_negotiation.cc:407] Connection header received
0807 15:10:03.434992 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:03.434996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:03.435065 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:03.435170 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:03.435452 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.436299 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.437381 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.438089 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.438231 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.438401 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:03.438588 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 15:10:03.438702 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 15:10:03.439853 25983 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 15:10:03.440342 26030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1be44596da5d4958b056b014ab975a39. Latest consensus state: current_term: 1 leader_uuid: "1be44596da5d4958b056b014ab975a39" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: NON_VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: true } } }
I20260807 15:10:03.440538 26030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [sys.catalog]: This master's current role is: LEARNER
W20260807 15:10:03.511251 26430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:03.511595 26430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:03.511644 26430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:03.515954 26430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:03.516083 26430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:03.516149 26430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:10:03.522058 26430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:43293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.67.keytab
--principal=oryx/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:43089
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=46813
--tserver_master_addrs=127.15.226.126:44957,127.15.226.125:45651,127.15.226.124:46831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:03.523391 26430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:03.524456 26430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:03.536099 26440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:03.536091 26439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:03.536091 26443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:03.536463 26430 server_base.cc:1034] running on GCE node
I20260807 15:10:03.537061 26430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:03.537842 26430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:03.539075 26430 hybrid_clock.cc:648] HybridClock initialized: now 1786115403539063 us; error 36 us; skew 500 ppm
Aug 07 15:10:03 dist-test-slave-3kk6 krb5kdc[24828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115403, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:03.542388 26430 init.cc:377] Logged in from keytab as oryx/127.15.226.67@KRBTEST.COM (short username oryx)
I20260807 15:10:03.543602 26430 webserver.cc:460] Webserver started at http://127.15.226.67:46813/ using document root <none> and password file <none>
I20260807 15:10:03.544332 26430 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:03.544464 26430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:03.549132 26430 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 15:10:03.551723 26452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:03.553345 26430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:10:03.553566 26430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "b85d9bf8304749cea743efe6d297ea65"
format_stamp: "Formatted at 2026-08-07 15:10:01 on dist-test-slave-3kk6"
server_key: "c2e4ab65e424f13ff03a3bd1f9723104"
server_key_iv: "fb7486795026f044877c0c359b1cb4dc"
server_key_version: "encryptionkey@0"
I20260807 15:10:03.554330 26430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:03.572988 26430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:03.577523 26430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:03.577800 26430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:03.578585 26430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:03.579767 26430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:03.579828 26430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:03.579881 26430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:03.579910 26430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:03.592932 26430 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:43089
I20260807 15:10:03.592962 26611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:43089 every 8 connection(s)
I20260807 15:10:03.594446 26430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:10:03.594918 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 26430
Aug 07 15:10:03 dist-test-slave-3kk6 krb5kdc[24828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115403, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.124@KRBTEST.COM
Aug 07 15:10:03 dist-test-slave-3kk6 krb5kdc[24828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115403, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM
I20260807 15:10:03.612375 24941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.599777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43866 (local address 127.15.226.126:44957)
0807 15:10:03.599910 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:03.599914 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:03.600459 (+ 545us) server_negotiation.cc:407] Connection header received
0807 15:10:03.600603 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:03.600606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:03.600672 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:03.600767 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:03.602366 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.603087 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.604286 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.604576 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.604632 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.606889 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:03.606894 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:03.606897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:03.606931 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:03.608571 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.609095 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.609101 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.609104 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.609203 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.609578 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.609587 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.609591 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.609853 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:03.609981 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:03.610439 (+ 458us) server_negotiation.cc:299] Negotiation successful
0807 15:10:03.610575 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 15:10:03.614004 26626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.598268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35096 (local address 127.15.226.125:45651)
0807 15:10:03.599452 (+ 1184us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:03.599457 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:03.599478 (+ 21us) server_negotiation.cc:407] Connection header received
0807 15:10:03.599532 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:03.599535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:03.599586 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:03.600657 (+ 1071us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:03.601866 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.602814 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.604924 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.605456 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.605581 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.607399 (+ 1818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:03.607404 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:03.607407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:03.607434 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:03.611069 (+ 3635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.611607 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.611610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.611612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.611724 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.612078 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.612084 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.612086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.612306 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:03.613299 (+ 993us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:03.613389 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 15:10:03.613810 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1091,"thread_start_us":262,"threads_started":1}
I20260807 15:10:03.617465 26233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.599794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:59265 (local address 127.15.226.124:46831)
0807 15:10:03.599972 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:03.599976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:03.601175 (+ 1199us) server_negotiation.cc:407] Connection header received
0807 15:10:03.601300 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:03.601303 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:03.601365 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:03.601446 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:03.603679 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.604570 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.607542 (+ 2972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.607961 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.608082 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.612355 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:03.612360 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:03.612363 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:03.612396 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:03.614361 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.615295 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.615300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.615305 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.615417 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.616031 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.616034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.616036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.616305 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:03.616508 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:03.617225 (+ 717us) server_negotiation.cc:299] Negotiation successful
0807 15:10:03.617353 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 15:10:03.619635 26633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.599779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:49597 (local address 127.15.226.126:44957)
0807 15:10:03.600163 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:03.600175 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:03.600194 (+ 19us) server_negotiation.cc:407] Connection header received
0807 15:10:03.601217 (+ 1023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:03.601222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:03.601301 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:03.601406 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:03.603592 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.604349 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.607347 (+ 2998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.607734 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.607858 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.614782 (+ 6924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:03.614787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:03.614791 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:03.614820 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:03.616551 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.618009 (+ 1458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.618012 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.618013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.618107 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.618502 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.618505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.618507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.618808 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:03.618954 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:03.619323 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 15:10:03.619463 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":140,"server-negotiator.queue_time_us":309,"thread_start_us":104,"threads_started":1}
I20260807 15:10:03.616662 24892 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 15:10:03.621481 26628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.597982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.67:49597)
0807 15:10:03.599129 (+ 1147us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:03.599195 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:03.600665 (+ 1470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:03.601828 (+ 1163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:03.601840 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:03.602396 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:03.603366 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.603408 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.604765 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.604768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.606587 (+ 1819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.606599 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.608217 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.608220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.608484 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.610561 (+ 2077us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:03.610579 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:03.613583 (+ 3004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:03.617466 (+ 3883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:03.617469 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:03.617471 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:03.617730 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:03.618211 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:03.618214 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:03.618217 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:03.618320 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:03.619008 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:03.619012 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:03.619111 (+ 99us) client_negotiation.cc:769] Sending connection context
0807 15:10:03.619222 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 15:10:03.620132 (+ 910us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"mutex_wait_us":845,"thread_start_us":800,"threads_started":1}
I20260807 15:10:03.621481 26629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.598461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:46831 (local address 127.15.226.67:59265)
0807 15:10:03.601039 (+ 2578us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:03.601053 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:03.601212 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:03.602965 (+ 1753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:03.602968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:03.603326 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:03.603558 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.603569 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.605703 (+ 2134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.605706 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.607253 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.607266 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.608118 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.608122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.608485 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.609456 (+ 971us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:03.609479 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:03.612093 (+ 2614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:03.614683 (+ 2590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:03.614692 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:03.614715 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:03.615149 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:03.615664 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:03.615670 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:03.615673 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:03.615898 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:03.616599 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:03.616606 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:03.616944 (+ 338us) client_negotiation.cc:769] Sending connection context
0807 15:10:03.617148 (+ 204us) client_negotiation.cc:240] Negotiation successful
0807 15:10:03.620082 (+ 2934us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2502,"mutex_wait_us":2505,"spinlock_wait_cycles":17792,"thread_start_us":164,"threads_started":1}
W20260807 15:10:03.622342 24892 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Aug 07 15:10:03 dist-test-slave-3kk6 krb5kdc[24828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115403, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.125@KRBTEST.COM
I20260807 15:10:03.623145 26617 heartbeater.cc:347] Connected to a master server at 127.15.226.126:44957
I20260807 15:10:03.623405 26617 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:03.624073 26617 heartbeater.cc:511] Master 127.15.226.126:44957 requested a full tablet report, sending...
I20260807 15:10:03.623142 26613 heartbeater.cc:347] Connected to a master server at 127.15.226.124:46831
I20260807 15:10:03.625137 26613 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:03.625330 26613 heartbeater.cc:511] Master 127.15.226.124:46831 requested a full tablet report, sending...
I20260807 15:10:03.625787 24891 ts_manager.cc:194] Re-registered known tserver with Master: b85d9bf8304749cea743efe6d297ea65 (127.15.226.67:43089)
I20260807 15:10:03.626255 25974 ts_manager.cc:194] Registered new tserver with Master: b85d9bf8304749cea743efe6d297ea65 (127.15.226.67:43089)
I20260807 15:10:03.626648 24891 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.226.67@KRBTEST.COM'} at 127.15.226.67:49597
I20260807 15:10:03.627646 26431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1be44596da5d4958b056b014ab975a39" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: NON_VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: true } } }
I20260807 15:10:03.627781 26431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:03.628660 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.598232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:35739 (local address 127.15.226.125:45651)
0807 15:10:03.598376 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:03.598381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:03.609074 (+ 10693us) server_negotiation.cc:407] Connection header received
0807 15:10:03.609398 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:03.609402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:03.609460 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:03.609522 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:03.611096 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.612063 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.620060 (+ 7997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.620307 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.620448 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.623952 (+ 3504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:03.623956 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:03.623959 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:03.623991 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:03.625859 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.626711 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.626715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.626717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.626832 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.627509 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.627513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.627517 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.627759 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:03.627856 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:03.628253 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 15:10:03.628438 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 15:10:03.629290 26632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.597982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:45651 (local address 127.15.226.67:35739)
0807 15:10:03.608937 (+ 10955us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:03.608951 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:03.609112 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:03.609689 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:03.609694 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:03.610587 (+ 893us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:03.610929 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.610941 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.612598 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.612604 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.619917 (+ 7313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.619929 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.620524 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.620529 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.620661 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.621266 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:03.621281 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:03.623755 (+ 2474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:03.626004 (+ 2249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:03.626007 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:03.626009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:03.626267 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:03.627191 (+ 924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:03.627196 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:03.627199 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:03.627377 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:03.627925 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:03.627929 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:03.628023 (+ 94us) client_negotiation.cc:769] Sending connection context
0807 15:10:03.628842 (+ 819us) client_negotiation.cc:240] Negotiation successful
0807 15:10:03.629041 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9960,"mutex_wait_us":1435,"thread_start_us":110,"threads_started":1}
I20260807 15:10:03.630187 26618 heartbeater.cc:347] Connected to a master server at 127.15.226.125:45651
I20260807 15:10:03.630299 26618 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:03.630533 26618 heartbeater.cc:511] Master 127.15.226.125:45651 requested a full tablet report, sending...
I20260807 15:10:03.631331 25710 ts_manager.cc:194] Registered new tserver with Master: b85d9bf8304749cea743efe6d297ea65 (127.15.226.67:43089)
I20260807 15:10:03.632977 26635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.601638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41220 (local address 127.15.226.124:46831)
0807 15:10:03.609959 (+ 8321us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:03.609972 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:03.609987 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:10:03.610026 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:03.610029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:03.610074 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:03.619114 (+ 9040us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:03.619245 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.623519 (+ 4274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.625435 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.625855 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.626458 (+ 603us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.627965 (+ 1507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:03.627968 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:03.627971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:03.627992 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:03.630463 (+ 2471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.631074 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.631077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.631079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.631173 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.631538 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.631544 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.631551 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.631812 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:03.631955 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:03.632680 (+ 725us) server_negotiation.cc:299] Negotiation successful
0807 15:10:03.632816 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8235,"thread_start_us":124,"threads_started":1}
I20260807 15:10:03.633493 26639 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39: attempting to promote NON_VOTER 7703a9b88b4a46eaa83a4207fe7bd66e to VOTER
I20260807 15:10:03.633977 26639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: false } }
I20260807 15:10:03.635603 25983 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 1 LEARNER]: Refusing update from remote peer 1be44596da5d4958b056b014ab975a39: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 15:10:03.635600 25722 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 1 FOLLOWER]: Refusing update from remote peer 1be44596da5d4958b056b014ab975a39: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 15:10:03.636029 26644 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 15:10:03.636193 26431 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 15:10:03.638269 26639 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 7703a9b88b4a46eaa83a4207fe7bd66e (127.15.226.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: false } } }
I20260807 15:10:03.638749 25722 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 7703a9b88b4a46eaa83a4207fe7bd66e (127.15.226.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: false } } }
I20260807 15:10:03.638870 25983 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 7703a9b88b4a46eaa83a4207fe7bd66e (127.15.226.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: false } } }
I20260807 15:10:03.640206 26639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1be44596da5d4958b056b014ab975a39" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: false } } }
I20260807 15:10:03.640331 26639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:03.640478 26639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1be44596da5d4958b056b014ab975a39" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: false } } }
I20260807 15:10:03.640571 26639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:03.640561 26431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1be44596da5d4958b056b014ab975a39" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: false } } }
I20260807 15:10:03.640704 26431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:03.641005 26030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1be44596da5d4958b056b014ab975a39" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: false } } }
I20260807 15:10:03.641124 26030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:10:03.642886 26651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1be44596da5d4958b056b014ab975a39" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: false } } }
I20260807 15:10:03.643057 26651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:10:03.658169 26661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.652192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:44821 (local address 127.0.0.1:37130)
0807 15:10:03.652505 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:03.652519 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:03.652616 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:03.653548 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:03.653552 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:03.653570 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:03.653664 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.653670 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.655059 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.655062 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.656362 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.656375 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.657702 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.657704 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.657775 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.657795 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 15:10:03.657906 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 15:10:03.657994 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":136,"threads_started":1}
I20260807 15:10:03.658259 26660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.651840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37130 (local address 127.15.226.66:44821)
0807 15:10:03.652307 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:03.652371 (+ 64us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:03.652957 (+ 586us) server_negotiation.cc:407] Connection header received
0807 15:10:03.653046 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:03.653051 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:03.653221 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:03.653334 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:03.653817 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.654943 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.656520 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.657399 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.657503 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.657645 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:03.657959 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 15:10:03.658068 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":109,"threads_started":1}
I20260807 15:10:03.658879 26655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.651625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:37605 (local address 127.0.0.1:53886)
0807 15:10:03.652018 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:03.652051 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:03.652171 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:03.653990 (+ 1819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:03.653993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:03.654011 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:03.654102 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.654108 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.655288 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.655292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.656261 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.656273 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.658452 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.658455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.658538 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.658561 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 15:10:03.658607 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 15:10:03.658722 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":242,"threads_started":1}
I20260807 15:10:03.660045 26433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.651103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:43089 (local address 127.0.0.1:54694)
0807 15:10:03.651240 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:03.651260 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:03.651366 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:03.653224 (+ 1858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:03.653231 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:03.653253 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:03.653393 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.653399 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.657490 (+ 4091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.657494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.658225 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.658234 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.659624 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.659629 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.659735 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.659754 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 15:10:03.659866 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 15:10:03.659951 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 15:10:03.659543 26659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.651840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53886 (local address 127.15.226.65:37605)
0807 15:10:03.653532 (+ 1692us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:03.653538 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:03.653562 (+ 24us) server_negotiation.cc:407] Connection header received
0807 15:10:03.653636 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:03.653642 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:03.653862 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:03.654015 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:03.654187 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.655171 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.656460 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.657360 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.659089 (+ 1729us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.659258 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:03.659309 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 15:10:03.659410 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1558,"thread_start_us":96,"threads_started":1}
I20260807 15:10:03.660181 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.651343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54694 (local address 127.15.226.67:43089)
0807 15:10:03.652591 (+ 1248us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:03.652618 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:03.652648 (+ 30us) server_negotiation.cc:407] Connection header received
0807 15:10:03.652744 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:03.652750 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:03.653019 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:03.653220 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:03.655436 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.657349 (+ 1913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.658512 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.659494 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.659644 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.659790 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:03.659907 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 15:10:03.660005 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1128,"thread_start_us":171,"threads_started":1}
I20260807 15:10:03.662288 26319 tablet_service.cc:1467] Processing CreateTablet for tablet 78a0050c890e4ba0a0c52b29c5b94311 (DEFAULT_TABLE table=test-table [id=d1013dab98e74feb98887425d6ffa9bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:03.663691 26319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78a0050c890e4ba0a0c52b29c5b94311. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:03.663333 26134 tablet_service.cc:1467] Processing CreateTablet for tablet 78a0050c890e4ba0a0c52b29c5b94311 (DEFAULT_TABLE table=test-table [id=d1013dab98e74feb98887425d6ffa9bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:03.664451 26134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78a0050c890e4ba0a0c52b29c5b94311. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:03.665362 26526 tablet_service.cc:1467] Processing CreateTablet for tablet 78a0050c890e4ba0a0c52b29c5b94311 (DEFAULT_TABLE table=test-table [id=d1013dab98e74feb98887425d6ffa9bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:03.666417 26526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78a0050c890e4ba0a0c52b29c5b94311. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:03.673943 26664 tablet_bootstrap.cc:492] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65: Bootstrap starting.
I20260807 15:10:03.674901 26662 tablet_bootstrap.cc:492] T 78a0050c890e4ba0a0c52b29c5b94311 P f448c8ea9dc5445496f57727ac1061bb: Bootstrap starting.
I20260807 15:10:03.676368 26664 tablet_bootstrap.cc:654] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:03.676767 26662 tablet_bootstrap.cc:654] T 78a0050c890e4ba0a0c52b29c5b94311 P f448c8ea9dc5445496f57727ac1061bb: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:03.677403 26664 log.cc:824] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:03.677665 26662 log.cc:824] T 78a0050c890e4ba0a0c52b29c5b94311 P f448c8ea9dc5445496f57727ac1061bb: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:03.678489 26665 tablet_bootstrap.cc:492] T 78a0050c890e4ba0a0c52b29c5b94311 P 60340eaffc12444aaf7a470a0634f9ac: Bootstrap starting.
I20260807 15:10:03.679833 26664 tablet_bootstrap.cc:492] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65: No bootstrap required, opened a new log
I20260807 15:10:03.679833 26662 tablet_bootstrap.cc:492] T 78a0050c890e4ba0a0c52b29c5b94311 P f448c8ea9dc5445496f57727ac1061bb: No bootstrap required, opened a new log
I20260807 15:10:03.680045 26662 ts_tablet_manager.cc:1397] T 78a0050c890e4ba0a0c52b29c5b94311 P f448c8ea9dc5445496f57727ac1061bb: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260807 15:10:03.680045 26664 ts_tablet_manager.cc:1397] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 15:10:03.681069 26665 tablet_bootstrap.cc:654] T 78a0050c890e4ba0a0c52b29c5b94311 P 60340eaffc12444aaf7a470a0634f9ac: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:03.682116 26665 log.cc:824] T 78a0050c890e4ba0a0c52b29c5b94311 P 60340eaffc12444aaf7a470a0634f9ac: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:03.684105 26665 tablet_bootstrap.cc:492] T 78a0050c890e4ba0a0c52b29c5b94311 P 60340eaffc12444aaf7a470a0634f9ac: No bootstrap required, opened a new log
I20260807 15:10:03.684345 26665 ts_tablet_manager.cc:1397] T 78a0050c890e4ba0a0c52b29c5b94311 P 60340eaffc12444aaf7a470a0634f9ac: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 15:10:03.685297 26662 raft_consensus.cc:348] T 78a0050c890e4ba0a0c52b29c5b94311 P f448c8ea9dc5445496f57727ac1061bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85d9bf8304749cea743efe6d297ea65" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43089 } } peers { permanent_uuid: "60340eaffc12444aaf7a470a0634f9ac" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37605 } } peers { permanent_uuid: "f448c8ea9dc5445496f57727ac1061bb" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44821 } }
I20260807 15:10:03.685297 26664 raft_consensus.cc:348] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85d9bf8304749cea743efe6d297ea65" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43089 } } peers { permanent_uuid: "60340eaffc12444aaf7a470a0634f9ac" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37605 } } peers { permanent_uuid: "f448c8ea9dc5445496f57727ac1061bb" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44821 } }
I20260807 15:10:03.685608 26664 raft_consensus.cc:374] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:03.685608 26662 raft_consensus.cc:374] T 78a0050c890e4ba0a0c52b29c5b94311 P f448c8ea9dc5445496f57727ac1061bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:03.685673 26662 raft_consensus.cc:729] T 78a0050c890e4ba0a0c52b29c5b94311 P f448c8ea9dc5445496f57727ac1061bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f448c8ea9dc5445496f57727ac1061bb, State: Initialized, Role: FOLLOWER
I20260807 15:10:03.685685 26664 raft_consensus.cc:729] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b85d9bf8304749cea743efe6d297ea65, State: Initialized, Role: FOLLOWER
I20260807 15:10:03.686209 26662 consensus_queue.cc:260] T 78a0050c890e4ba0a0c52b29c5b94311 P f448c8ea9dc5445496f57727ac1061bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85d9bf8304749cea743efe6d297ea65" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43089 } } peers { permanent_uuid: "60340eaffc12444aaf7a470a0634f9ac" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37605 } } peers { permanent_uuid: "f448c8ea9dc5445496f57727ac1061bb" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44821 } }
I20260807 15:10:03.686201 26664 consensus_queue.cc:260] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85d9bf8304749cea743efe6d297ea65" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43089 } } peers { permanent_uuid: "60340eaffc12444aaf7a470a0634f9ac" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37605 } } peers { permanent_uuid: "f448c8ea9dc5445496f57727ac1061bb" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44821 } }
I20260807 15:10:03.687429 26664 ts_tablet_manager.cc:1428] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260807 15:10:03.687429 26662 ts_tablet_manager.cc:1428] T 78a0050c890e4ba0a0c52b29c5b94311 P f448c8ea9dc5445496f57727ac1061bb: Time spent starting tablet: real 0.007s user 0.002s sys 0.006s
I20260807 15:10:03.687668 26424 heartbeater.cc:503] Master 127.15.226.126:44957 was elected leader, sending a full tablet report...
I20260807 15:10:03.688208 26617 heartbeater.cc:503] Master 127.15.226.126:44957 was elected leader, sending a full tablet report...
I20260807 15:10:03.693397 26665 raft_consensus.cc:348] T 78a0050c890e4ba0a0c52b29c5b94311 P 60340eaffc12444aaf7a470a0634f9ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85d9bf8304749cea743efe6d297ea65" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43089 } } peers { permanent_uuid: "60340eaffc12444aaf7a470a0634f9ac" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37605 } } peers { permanent_uuid: "f448c8ea9dc5445496f57727ac1061bb" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44821 } }
I20260807 15:10:03.693785 26665 raft_consensus.cc:374] T 78a0050c890e4ba0a0c52b29c5b94311 P 60340eaffc12444aaf7a470a0634f9ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:03.693854 26665 raft_consensus.cc:729] T 78a0050c890e4ba0a0c52b29c5b94311 P 60340eaffc12444aaf7a470a0634f9ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60340eaffc12444aaf7a470a0634f9ac, State: Initialized, Role: FOLLOWER
I20260807 15:10:03.694372 26665 consensus_queue.cc:260] T 78a0050c890e4ba0a0c52b29c5b94311 P 60340eaffc12444aaf7a470a0634f9ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85d9bf8304749cea743efe6d297ea65" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43089 } } peers { permanent_uuid: "60340eaffc12444aaf7a470a0634f9ac" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37605 } } peers { permanent_uuid: "f448c8ea9dc5445496f57727ac1061bb" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44821 } }
I20260807 15:10:03.695214 26230 heartbeater.cc:503] Master 127.15.226.126:44957 was elected leader, sending a full tablet report...
I20260807 15:10:03.695731 26665 ts_tablet_manager.cc:1428] T 78a0050c890e4ba0a0c52b29c5b94311 P 60340eaffc12444aaf7a470a0634f9ac: Time spent starting tablet: real 0.011s user 0.004s sys 0.004s
I20260807 15:10:03.797803 26671 raft_consensus.cc:482] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:10:03.798082 26671 raft_consensus.cc:504] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85d9bf8304749cea743efe6d297ea65" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43089 } } peers { permanent_uuid: "60340eaffc12444aaf7a470a0634f9ac" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37605 } } peers { permanent_uuid: "f448c8ea9dc5445496f57727ac1061bb" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44821 } }
I20260807 15:10:03.799461 26671 leader_election.cc:288] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60340eaffc12444aaf7a470a0634f9ac (127.15.226.65:37605), f448c8ea9dc5445496f57727ac1061bb (127.15.226.66:44821)
I20260807 15:10:03.804060 26660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.799908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:57849 (local address 127.15.226.66:44821)
0807 15:10:03.800037 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:03.800043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:03.800097 (+ 54us) server_negotiation.cc:407] Connection header received
0807 15:10:03.800293 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:03.800295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:03.800339 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:03.800402 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:03.800721 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.801379 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.802710 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.803338 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.803480 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.803526 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:03.803816 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 15:10:03.803937 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 15:10:03.804095 26659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.800063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:54673 (local address 127.15.226.65:37605)
0807 15:10:03.800207 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:03.800214 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:03.800248 (+ 34us) server_negotiation.cc:407] Connection header received
0807 15:10:03.800313 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:03.800315 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:03.800355 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:03.800446 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:03.800770 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.801358 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.802863 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.803392 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.803510 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.803564 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:03.803910 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 15:10:03.803986 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 15:10:03.804054 26632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.799844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:37605 (local address 127.15.226.67:54673)
0807 15:10:03.800012 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:03.800040 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:03.800156 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:03.800474 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:03.800477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:03.800494 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:03.800617 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.800623 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.801479 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.801482 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.802580 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.802590 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.803661 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.803664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.803772 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.803797 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 15:10:03.803866 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 15:10:03.803917 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":67,"spinlock_wait_cycles":6016}
I20260807 15:10:03.804054 26628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.799756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:44821 (local address 127.15.226.67:57849)
0807 15:10:03.800012 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:03.800031 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:03.800161 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:03.800443 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:03.800446 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:03.800462 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:03.800606 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.800613 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.801522 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.801525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.802545 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:03.802555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.803466 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.803471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:03.803675 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.803699 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 15:10:03.803795 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 15:10:03.803885 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"spinlock_wait_cycles":14464}
I20260807 15:10:03.805003 26349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78a0050c890e4ba0a0c52b29c5b94311" candidate_uuid: "b85d9bf8304749cea743efe6d297ea65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f448c8ea9dc5445496f57727ac1061bb" is_pre_election: true
I20260807 15:10:03.805037 26160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78a0050c890e4ba0a0c52b29c5b94311" candidate_uuid: "b85d9bf8304749cea743efe6d297ea65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60340eaffc12444aaf7a470a0634f9ac" is_pre_election: true
I20260807 15:10:03.805397 26160 raft_consensus.cc:2454] T 78a0050c890e4ba0a0c52b29c5b94311 P 60340eaffc12444aaf7a470a0634f9ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b85d9bf8304749cea743efe6d297ea65 in term 0.
I20260807 15:10:03.805423 26349 raft_consensus.cc:2454] T 78a0050c890e4ba0a0c52b29c5b94311 P f448c8ea9dc5445496f57727ac1061bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b85d9bf8304749cea743efe6d297ea65 in term 0.
I20260807 15:10:03.805930 26457 leader_election.cc:302] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60340eaffc12444aaf7a470a0634f9ac, b85d9bf8304749cea743efe6d297ea65; no voters:
I20260807 15:10:03.806246 26671 raft_consensus.cc:2781] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:10:03.806335 26671 raft_consensus.cc:482] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:10:03.806370 26671 raft_consensus.cc:3037] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:03.807763 26671 raft_consensus.cc:504] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85d9bf8304749cea743efe6d297ea65" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43089 } } peers { permanent_uuid: "60340eaffc12444aaf7a470a0634f9ac" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37605 } } peers { permanent_uuid: "f448c8ea9dc5445496f57727ac1061bb" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44821 } }
I20260807 15:10:03.808185 26671 leader_election.cc:288] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65 [CANDIDATE]: Term 1 election: Requested vote from peers 60340eaffc12444aaf7a470a0634f9ac (127.15.226.65:37605), f448c8ea9dc5445496f57727ac1061bb (127.15.226.66:44821)
I20260807 15:10:03.808634 26349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78a0050c890e4ba0a0c52b29c5b94311" candidate_uuid: "b85d9bf8304749cea743efe6d297ea65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f448c8ea9dc5445496f57727ac1061bb"
I20260807 15:10:03.808804 26349 raft_consensus.cc:3037] T 78a0050c890e4ba0a0c52b29c5b94311 P f448c8ea9dc5445496f57727ac1061bb [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:03.808812 26160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78a0050c890e4ba0a0c52b29c5b94311" candidate_uuid: "b85d9bf8304749cea743efe6d297ea65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60340eaffc12444aaf7a470a0634f9ac"
I20260807 15:10:03.808979 26160 raft_consensus.cc:3037] T 78a0050c890e4ba0a0c52b29c5b94311 P 60340eaffc12444aaf7a470a0634f9ac [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:03.810222 26349 raft_consensus.cc:2454] T 78a0050c890e4ba0a0c52b29c5b94311 P f448c8ea9dc5445496f57727ac1061bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b85d9bf8304749cea743efe6d297ea65 in term 1.
I20260807 15:10:03.810231 26160 raft_consensus.cc:2454] T 78a0050c890e4ba0a0c52b29c5b94311 P 60340eaffc12444aaf7a470a0634f9ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b85d9bf8304749cea743efe6d297ea65 in term 1.
I20260807 15:10:03.810585 26454 leader_election.cc:302] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b85d9bf8304749cea743efe6d297ea65, f448c8ea9dc5445496f57727ac1061bb; no voters:
I20260807 15:10:03.810842 26671 raft_consensus.cc:2781] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:03.811147 26671 raft_consensus.cc:686] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65 [term 1 LEADER]: Becoming Leader. State: Replica: b85d9bf8304749cea743efe6d297ea65, State: Running, Role: LEADER
I20260807 15:10:03.811607 26671 consensus_queue.cc:237] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85d9bf8304749cea743efe6d297ea65" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43089 } } peers { permanent_uuid: "60340eaffc12444aaf7a470a0634f9ac" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37605 } } peers { permanent_uuid: "f448c8ea9dc5445496f57727ac1061bb" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44821 } }
I20260807 15:10:03.815567 24891 catalog_manager.cc:5495] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65 reported cstate change: term changed from 0 to 1, leader changed from <none> to b85d9bf8304749cea743efe6d297ea65 (127.15.226.67). New cstate: current_term: 1 leader_uuid: "b85d9bf8304749cea743efe6d297ea65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b85d9bf8304749cea743efe6d297ea65" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60340eaffc12444aaf7a470a0634f9ac" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f448c8ea9dc5445496f57727ac1061bb" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44821 } health_report { overall_health: UNKNOWN } } }
W20260807 15:10:03.834638 26426 tablet.cc:2369] T 78a0050c890e4ba0a0c52b29c5b94311 P f448c8ea9dc5445496f57727ac1061bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:10:03.838526 26232 tablet.cc:2369] T 78a0050c890e4ba0a0c52b29c5b94311 P 60340eaffc12444aaf7a470a0634f9ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:10:03.847769 26619 tablet.cc:2369] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:10:03.849402 26654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.845465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54708 (local address 127.15.226.67:43089)
0807 15:10:03.845698 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:03.845702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:03.845714 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:10:03.845794 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:03.845796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:03.845853 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:03.845945 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:10:03.846218 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.846791 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.847677 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.847994 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.848102 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.848469 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:10:03.848903 (+ 434us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:10:03.849001 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:03.849126 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 15:10:03.849186 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
I20260807 15:10:03.856022 26160 raft_consensus.cc:1261] T 78a0050c890e4ba0a0c52b29c5b94311 P 60340eaffc12444aaf7a470a0634f9ac [term 1 FOLLOWER]: Refusing update from remote peer b85d9bf8304749cea743efe6d297ea65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:03.856029 26349 raft_consensus.cc:1261] T 78a0050c890e4ba0a0c52b29c5b94311 P f448c8ea9dc5445496f57727ac1061bb [term 1 FOLLOWER]: Refusing update from remote peer b85d9bf8304749cea743efe6d297ea65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:03.856817 26671 consensus_queue.cc:1035] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f448c8ea9dc5445496f57727ac1061bb" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:03.857033 26673 consensus_queue.cc:1035] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60340eaffc12444aaf7a470a0634f9ac" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:03.866050 26678 mvcc.cc:205] Tried to move back new op lower bound from 7315928694185758720 to 7315928694015614976. Current Snapshot: MvccSnapshot[applied={T|T < 7315928694185758720}]
I20260807 15:10:03.868453 26679 mvcc.cc:205] Tried to move back new op lower bound from 7315928694185758720 to 7315928694015614976. Current Snapshot: MvccSnapshot[applied={T|T < 7315928694185758720}]
I20260807 15:10:03.872195 26680 mvcc.cc:205] Tried to move back new op lower bound from 7315928694185758720 to 7315928694015614976. Current Snapshot: MvccSnapshot[applied={T|T < 7315928694185758720}]
I20260807 15:10:03.879720 24892 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:03.879966 24892 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:03.885730 24892 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1be44596da5d4958b056b014ab975a39: Sending DeleteTablet for 3 replicas of tablet 78a0050c890e4ba0a0c52b29c5b94311
I20260807 15:10:03.886691 26526 tablet_service.cc:1514] Processing DeleteTablet for tablet 78a0050c890e4ba0a0c52b29c5b94311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:03 UTC) from {username='oryx', principal='oryx/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:54694
I20260807 15:10:03.886771 26319 tablet_service.cc:1514] Processing DeleteTablet for tablet 78a0050c890e4ba0a0c52b29c5b94311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:03 UTC) from {username='oryx', principal='oryx/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:37130
I20260807 15:10:03.886983 26134 tablet_service.cc:1514] Processing DeleteTablet for tablet 78a0050c890e4ba0a0c52b29c5b94311 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:03 UTC) from {username='oryx', principal='oryx/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:53886
I20260807 15:10:03.887518 26696 tablet_replica.cc:330] T 78a0050c890e4ba0a0c52b29c5b94311 P 60340eaffc12444aaf7a470a0634f9ac: stopping tablet replica
I20260807 15:10:03.887966 26696 raft_consensus.cc:2229] T 78a0050c890e4ba0a0c52b29c5b94311 P 60340eaffc12444aaf7a470a0634f9ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:10:03.888022 26693 tablet_replica.cc:330] T 78a0050c890e4ba0a0c52b29c5b94311 P f448c8ea9dc5445496f57727ac1061bb: stopping tablet replica
I20260807 15:10:03.888319 26693 raft_consensus.cc:2229] T 78a0050c890e4ba0a0c52b29c5b94311 P f448c8ea9dc5445496f57727ac1061bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:10:03.888540 26696 raft_consensus.cc:2258] T 78a0050c890e4ba0a0c52b29c5b94311 P 60340eaffc12444aaf7a470a0634f9ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:03.888600 26693 raft_consensus.cc:2258] T 78a0050c890e4ba0a0c52b29c5b94311 P f448c8ea9dc5445496f57727ac1061bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:03.889037 26694 tablet_replica.cc:330] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65: stopping tablet replica
I20260807 15:10:03.889354 26694 raft_consensus.cc:2229] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:10:03.889807 26694 raft_consensus.cc:2258] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:03.890784 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 24847
I20260807 15:10:03.912500 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.889322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35100 (local address 127.15.226.125:45651)
0807 15:10:03.889577 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:03.889581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:03.890800 (+ 1219us) server_negotiation.cc:407] Connection header received
0807 15:10:03.901324 (+ 10524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:03.901330 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:03.901402 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:03.901463 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:03.903128 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.903879 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.905136 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.905472 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.905557 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.908338 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:03.908345 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:03.908351 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:03.908389 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:03.910328 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.910936 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.910940 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.910942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.911037 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.911372 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.911375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.911377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.911636 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:03.911788 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:03.912174 (+ 386us) server_negotiation.cc:299] Negotiation successful
0807 15:10:03.912318 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175}
I20260807 15:10:03.921000 26693 ts_tablet_manager.cc:1905] T 78a0050c890e4ba0a0c52b29c5b94311 P f448c8ea9dc5445496f57727ac1061bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:03.923656 26635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.889445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41222 (local address 127.15.226.124:46831)
0807 15:10:03.889576 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:03.889580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:03.913601 (+ 24021us) server_negotiation.cc:407] Connection header received
0807 15:10:03.913812 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:03.913818 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:03.913913 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:03.913977 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:03.914958 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.915853 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.917045 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.917413 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.917493 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.919583 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:03.919588 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:03.919591 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:03.919624 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:03.921555 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.922114 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.922118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.922120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.922215 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.922522 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.922527 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.922531 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.922795 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:03.922938 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:03.923303 (+ 365us) server_negotiation.cc:299] Negotiation successful
0807 15:10:03.923470 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 15:10:03.925163 26693 ts_tablet_manager.cc:1918] T 78a0050c890e4ba0a0c52b29c5b94311 P f448c8ea9dc5445496f57727ac1061bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:03.925309 26693 log.cc:1196] T 78a0050c890e4ba0a0c52b29c5b94311 P f448c8ea9dc5445496f57727ac1061bb: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/78a0050c890e4ba0a0c52b29c5b94311
I20260807 15:10:03.925700 26693 ts_tablet_manager.cc:1939] T 78a0050c890e4ba0a0c52b29c5b94311 P f448c8ea9dc5445496f57727ac1061bb: Deleting consensus metadata
I20260807 15:10:03.927978 26696 ts_tablet_manager.cc:1905] T 78a0050c890e4ba0a0c52b29c5b94311 P 60340eaffc12444aaf7a470a0634f9ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:03.928928 26694 ts_tablet_manager.cc:1905] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 15:10:03.929889 26257 connection.cc:535] server connection from 127.0.0.1:37130 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 15:10:03.932979 26694 ts_tablet_manager.cc:1918] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:03.933109 26694 log.cc:1196] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/78a0050c890e4ba0a0c52b29c5b94311
I20260807 15:10:03.933518 26694 ts_tablet_manager.cc:1939] T 78a0050c890e4ba0a0c52b29c5b94311 P b85d9bf8304749cea743efe6d297ea65: Deleting consensus metadata
W20260807 15:10:03.934767 26455 connection.cc:441] server connection from 127.0.0.1:54694 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54694 (request call id 1) could send its response
I20260807 15:10:03.937227 26696 ts_tablet_manager.cc:1918] T 78a0050c890e4ba0a0c52b29c5b94311 P 60340eaffc12444aaf7a470a0634f9ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:03.937366 26696 log.cc:1196] T 78a0050c890e4ba0a0c52b29c5b94311 P 60340eaffc12444aaf7a470a0634f9ac: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-0/wal/wals/78a0050c890e4ba0a0c52b29c5b94311
I20260807 15:10:03.937768 26696 ts_tablet_manager.cc:1939] T 78a0050c890e4ba0a0c52b29c5b94311 P 60340eaffc12444aaf7a470a0634f9ac: Deleting consensus metadata
W20260807 15:10:03.939220 26066 connection.cc:441] server connection from 127.0.0.1:53886 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53886 (request call id 1) could send its response
I20260807 15:10:03.945716 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.934561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35110 (local address 127.15.226.125:45651)
0807 15:10:03.934721 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:03.934726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:03.934811 (+ 85us) server_negotiation.cc:407] Connection header received
0807 15:10:03.935039 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:03.935043 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:03.935102 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:03.935164 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:03.936476 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.937417 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.938844 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.939203 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.939296 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.941426 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:03.941430 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:03.941433 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:03.941465 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:03.943453 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.943997 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.944002 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.944004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.944105 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.944608 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.944612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.944614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.944915 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:03.945053 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:03.945358 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 15:10:03.945500 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 15:10:03.946126 26635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:03.934063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41232 (local address 127.15.226.124:46831)
0807 15:10:03.934209 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:03.934216 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:03.935301 (+ 1085us) server_negotiation.cc:407] Connection header received
0807 15:10:03.935486 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:03.935492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:03.935549 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:03.935608 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:03.936602 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.937418 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.940014 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:03.940397 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:03.940521 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:03.942093 (+ 1572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:03.942097 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:03.942099 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:03.942136 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:03.944126 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.944617 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.944621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.944623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.944718 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:03.945084 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:03.945088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:03.945089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:03.945331 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:03.945431 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:03.945821 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 15:10:03.945967 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 15:10:04.869881 26706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:04.869274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.65:44435)
0807 15:10:04.869635 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:04.869752 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":97,"threads_started":1}
W20260807 15:10:04.870358 26230 heartbeater.cc:644] Failed to heartbeat to 127.15.226.126:44957 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
I20260807 15:10:04.870704 26708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:04.869892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.67:45061)
0807 15:10:04.870472 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:04.870573 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":88,"threads_started":1}
W20260807 15:10:04.870978 26617 heartbeater.cc:644] Failed to heartbeat to 127.15.226.126:44957 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
I20260807 15:10:04.871526 26709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:04.870804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.66:41669)
0807 15:10:04.871299 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:04.871429 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":76,"threads_started":1}
W20260807 15:10:04.871899 26424 heartbeater.cc:644] Failed to heartbeat to 127.15.226.126:44957 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
I20260807 15:10:05.385934 26710 raft_consensus.cc:482] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1be44596da5d4958b056b014ab975a39)
I20260807 15:10:05.386016 26713 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1be44596da5d4958b056b014ab975a39)
I20260807 15:10:05.386128 26710 raft_consensus.cc:504] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: false } }
I20260807 15:10:05.386179 26713 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: false } }
I20260807 15:10:05.386925 26710 leader_election.cc:288] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1be44596da5d4958b056b014ab975a39 (127.15.226.126:44957), 7703a9b88b4a46eaa83a4207fe7bd66e (127.15.226.124:46831)
I20260807 15:10:05.387483 26713 leader_election.cc:288] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1be44596da5d4958b056b014ab975a39 (127.15.226.126:44957), 582deee640a1481cbf7a09c88c6269f7 (127.15.226.125:45651)
I20260807 15:10:05.387889 26714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:05.387123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.0.0.1:44398)
0807 15:10:05.387666 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:05.387772 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":100,"threads_started":1}
I20260807 15:10:05.387916 26717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:05.387176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.0.0.1:44404)
0807 15:10:05.387712 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:05.387806 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":71,"threads_started":1}
W20260807 15:10:05.388149 25956 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
W20260807 15:10:05.388207 25691 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
I20260807 15:10:05.389249 26717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:05.388970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.0.0.1:44406)
0807 15:10:05.389104 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:05.389157 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 15:10:05.389292 26714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:05.389047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.0.0.1:44408)
0807 15:10:05.389177 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:05.389225 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260807 15:10:05.389394 25956 leader_election.cc:334] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1be44596da5d4958b056b014ab975a39 (127.15.226.126:44957): Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
W20260807 15:10:05.389556 25691 leader_election.cc:334] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1be44596da5d4958b056b014ab975a39 (127.15.226.126:44957): Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
I20260807 15:10:05.392016 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:05.387710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:45651 (local address 127.0.0.1:35116)
0807 15:10:05.388057 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:05.388071 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:05.388185 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:05.388621 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:05.388625 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:05.388647 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:05.388767 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:05.388776 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:05.389861 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:05.389868 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:05.390775 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:05.390786 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:05.391441 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:05.391445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:05.391569 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:05.391633 (+ 64us) client_negotiation.cc:769] Sending connection context
0807 15:10:05.391780 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 15:10:05.391862 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":41,"thread_start_us":70,"threads_started":1}
I20260807 15:10:05.392011 26715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:05.387195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:46831 (local address 127.0.0.1:41240)
0807 15:10:05.387670 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:05.387700 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:05.387829 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:05.388626 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:05.388631 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:05.388653 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:05.388778 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:05.388785 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:05.389936 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:05.389941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:05.390849 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:05.390859 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:05.391551 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:05.391555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:05.391644 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:05.391666 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 15:10:05.391755 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 15:10:05.391829 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":74,"threads_started":1}
I20260807 15:10:05.392014 26724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:05.388012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35116 (local address 127.15.226.125:45651)
0807 15:10:05.388258 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:05.388263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:05.388280 (+ 17us) server_negotiation.cc:407] Connection header received
0807 15:10:05.388332 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:05.388335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:05.388385 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:05.388486 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:05.388923 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:05.389724 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:05.390905 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:05.391314 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:05.391447 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:05.391502 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:05.391782 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 15:10:05.391865 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":66,"threads_started":1}
I20260807 15:10:05.392493 26718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:05.387425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41240 (local address 127.15.226.124:46831)
0807 15:10:05.388358 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:05.388363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:05.388377 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:10:05.388427 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:05.388432 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:05.388486 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:05.388899 (+ 413us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:05.389006 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:05.389817 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:05.390979 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:05.391413 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:05.391610 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:05.391664 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:05.392240 (+ 576us) server_negotiation.cc:299] Negotiation successful
0807 15:10:05.392327 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":875,"thread_start_us":75,"threads_started":1}
I20260807 15:10:05.392762 25722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "582deee640a1481cbf7a09c88c6269f7" is_pre_election: true
I20260807 15:10:05.392970 25722 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7703a9b88b4a46eaa83a4207fe7bd66e in term 1.
I20260807 15:10:05.392975 25983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "582deee640a1481cbf7a09c88c6269f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" is_pre_election: true
I20260807 15:10:05.393275 25983 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 582deee640a1481cbf7a09c88c6269f7 in term 1.
I20260807 15:10:05.393299 25954 leader_election.cc:302] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 582deee640a1481cbf7a09c88c6269f7, 7703a9b88b4a46eaa83a4207fe7bd66e; no voters: 1be44596da5d4958b056b014ab975a39
I20260807 15:10:05.393509 26713 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 15:10:05.393571 26713 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1be44596da5d4958b056b014ab975a39)
I20260807 15:10:05.393610 26713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:10:05.393807 25690 leader_election.cc:302] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 582deee640a1481cbf7a09c88c6269f7, 7703a9b88b4a46eaa83a4207fe7bd66e; no voters: 1be44596da5d4958b056b014ab975a39
I20260807 15:10:05.393982 26710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 15:10:05.394032 26710 raft_consensus.cc:482] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1be44596da5d4958b056b014ab975a39)
I20260807 15:10:05.394075 26710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:10:05.394980 26713 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: false } }
I20260807 15:10:05.395321 26710 raft_consensus.cc:504] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: false } }
I20260807 15:10:05.395439 26713 leader_election.cc:288] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [CANDIDATE]: Term 2 election: Requested vote from peers 1be44596da5d4958b056b014ab975a39 (127.15.226.126:44957), 582deee640a1481cbf7a09c88c6269f7 (127.15.226.125:45651)
I20260807 15:10:05.395709 26710 leader_election.cc:288] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [CANDIDATE]: Term 2 election: Requested vote from peers 1be44596da5d4958b056b014ab975a39 (127.15.226.126:44957), 7703a9b88b4a46eaa83a4207fe7bd66e (127.15.226.124:46831)
I20260807 15:10:05.396111 25983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "582deee640a1481cbf7a09c88c6269f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e"
I20260807 15:10:05.396170 26715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:05.395899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.0.0.1:44426)
0807 15:10:05.396024 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:05.396090 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 15:10:05.396149 25722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "582deee640a1481cbf7a09c88c6269f7"
I20260807 15:10:05.396301 25983 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 582deee640a1481cbf7a09c88c6269f7 in current term 2: Already voted for candidate 7703a9b88b4a46eaa83a4207fe7bd66e in this term.
I20260807 15:10:05.396335 25722 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7703a9b88b4a46eaa83a4207fe7bd66e in current term 2: Already voted for candidate 582deee640a1481cbf7a09c88c6269f7 in this term.
I20260807 15:10:05.396868 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:05.395578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.0.0.1:44424)
0807 15:10:05.396737 (+ 1159us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:05.396795 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1073}
I20260807 15:10:05.396919 26715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:05.396683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.0.0.1:44430)
0807 15:10:05.396800 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:05.396856 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 15:10:05.397135 25691 leader_election.cc:334] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1be44596da5d4958b056b014ab975a39 (127.15.226.126:44957): Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
I20260807 15:10:05.397236 25691 leader_election.cc:302] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 582deee640a1481cbf7a09c88c6269f7; no voters: 1be44596da5d4958b056b014ab975a39, 7703a9b88b4a46eaa83a4207fe7bd66e
I20260807 15:10:05.397404 26710 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 15:10:05.397630 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:05.397361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.0.0.1:44442)
0807 15:10:05.397470 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:05.397565 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 15:10:05.397742 25956 leader_election.cc:334] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1be44596da5d4958b056b014ab975a39 (127.15.226.126:44957): Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
I20260807 15:10:05.397840 25956 leader_election.cc:302] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7703a9b88b4a46eaa83a4207fe7bd66e; no voters: 1be44596da5d4958b056b014ab975a39, 582deee640a1481cbf7a09c88c6269f7
I20260807 15:10:05.397989 26713 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260807 15:10:05.871831 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:05.871165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.65:43757)
0807 15:10:05.871588 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:05.871666 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":122,"threads_started":1}
I20260807 15:10:05.872346 26730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:05.871533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.67:53407)
0807 15:10:05.871895 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:05.871965 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":93,"threads_started":1}
I20260807 15:10:05.873050 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:05.872444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.66:56943)
0807 15:10:05.872886 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:05.872953 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":108,"threads_started":1}
I20260807 15:10:06.873517 26732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:06.872891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.65:48175)
0807 15:10:06.873273 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:06.873378 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":104,"threads_started":1}
I20260807 15:10:06.874197 26733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:06.872979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.67:35929)
0807 15:10:06.873292 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:06.873367 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":78,"threads_started":1}
I20260807 15:10:06.874816 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:06.874006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.66:44517)
0807 15:10:06.874614 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:06.874709 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":78,"threads_started":1}
I20260807 15:10:07.191387 26736 raft_consensus.cc:482] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:10:07.191568 26736 raft_consensus.cc:504] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: false } }
I20260807 15:10:07.192233 26736 leader_election.cc:288] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1be44596da5d4958b056b014ab975a39 (127.15.226.126:44957), 7703a9b88b4a46eaa83a4207fe7bd66e (127.15.226.124:46831)
I20260807 15:10:07.192857 25983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "582deee640a1481cbf7a09c88c6269f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" is_pre_election: true
I20260807 15:10:07.192932 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.192438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.0.0.1:44688)
0807 15:10:07.192696 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:07.192843 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":74,"threads_started":1}
I20260807 15:10:07.193005 25983 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 582deee640a1481cbf7a09c88c6269f7 in term 2.
I20260807 15:10:07.193583 25690 leader_election.cc:302] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 582deee640a1481cbf7a09c88c6269f7, 7703a9b88b4a46eaa83a4207fe7bd66e; no voters:
I20260807 15:10:07.193796 26736 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260807 15:10:07.193862 26736 raft_consensus.cc:482] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:10:07.193910 26736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:10:07.193987 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.193737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.0.0.1:44698)
0807 15:10:07.193853 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:07.193917 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 15:10:07.194077 25691 leader_election.cc:334] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1be44596da5d4958b056b014ab975a39 (127.15.226.126:44957): Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
I20260807 15:10:07.195329 26736 raft_consensus.cc:504] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: false } }
I20260807 15:10:07.195801 26736 leader_election.cc:288] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [CANDIDATE]: Term 3 election: Requested vote from peers 1be44596da5d4958b056b014ab975a39 (127.15.226.126:44957), 7703a9b88b4a46eaa83a4207fe7bd66e (127.15.226.124:46831)
I20260807 15:10:07.196199 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.195920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.0.0.1:44712)
0807 15:10:07.196074 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:07.196129 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 15:10:07.196399 25983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "582deee640a1481cbf7a09c88c6269f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e"
I20260807 15:10:07.196574 25983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:10:07.196956 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.196727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.0.0.1:44728)
0807 15:10:07.196849 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:07.196897 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 15:10:07.197055 25691 leader_election.cc:334] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 1be44596da5d4958b056b014ab975a39 (127.15.226.126:44957): Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
I20260807 15:10:07.197687 25983 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 582deee640a1481cbf7a09c88c6269f7 in term 3.
I20260807 15:10:07.198041 25690 leader_election.cc:302] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 582deee640a1481cbf7a09c88c6269f7, 7703a9b88b4a46eaa83a4207fe7bd66e; no voters: 1be44596da5d4958b056b014ab975a39
I20260807 15:10:07.198249 26736 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 15:10:07.198462 26736 raft_consensus.cc:686] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [term 3 LEADER]: Becoming Leader. State: Replica: 582deee640a1481cbf7a09c88c6269f7, State: Running, Role: LEADER
I20260807 15:10:07.198876 26736 consensus_queue.cc:237] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: false } }
I20260807 15:10:07.200099 26740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 582deee640a1481cbf7a09c88c6269f7. Latest consensus state: current_term: 3 leader_uuid: "582deee640a1481cbf7a09c88c6269f7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: false } } }
I20260807 15:10:07.200242 26740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:07.200534 26745 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:07.203394 26745 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:10:07.203835 26745 catalog_manager.cc:1259] Loaded cluster ID: 3115c820c8e740de80daae454f6c5f1c
I20260807 15:10:07.203907 26745 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:10:07.204491 26745 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:07.204887 26745 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7: Loaded TSK: 0
I20260807 15:10:07.205605 26745 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:10:07.264094 25710 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35110:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:10:07.267391 25710 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:10:07.270815 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.270494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.0.0.1:44736)
0807 15:10:07.270627 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:07.270735 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 15:10:07.270978 25983 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [term 3 FOLLOWER]: Refusing update from remote peer 582deee640a1481cbf7a09c88c6269f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260807 15:10:07.271565 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.271311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.0.0.1:44752)
0807 15:10:07.271435 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:07.271493 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 15:10:07.271581 26740 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260807 15:10:07.271955 25691 consensus_peers.cc:487] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 -> Peer 1be44596da5d4958b056b014ab975a39 (127.15.226.126:44957): Couldn't send request to peer 1be44596da5d4958b056b014ab975a39. Status: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 15:10:07.273307 26748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 582deee640a1481cbf7a09c88c6269f7. Latest consensus state: current_term: 3 leader_uuid: "582deee640a1481cbf7a09c88c6269f7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: false } } }
I20260807 15:10:07.273442 26748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:10:07.274763 26740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "582deee640a1481cbf7a09c88c6269f7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: false } } }
I20260807 15:10:07.274920 26740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:07.275558 26748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "582deee640a1481cbf7a09c88c6269f7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1be44596da5d4958b056b014ab975a39" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44957 } } peers { permanent_uuid: "582deee640a1481cbf7a09c88c6269f7" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45651 } attrs { promote: false } } peers { permanent_uuid: "7703a9b88b4a46eaa83a4207fe7bd66e" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 46831 } attrs { promote: false } } }
I20260807 15:10:07.275687 26748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7703a9b88b4a46eaa83a4207fe7bd66e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:10:07.277473 26749 mvcc.cc:205] Tried to move back new op lower bound from 7315928708175642624 to 7315928707889659904. Current Snapshot: MvccSnapshot[applied={T|T < 7315928708175642624 or (T in {7315928708175642624})}]
I20260807 15:10:07.285758 26737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.281312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:44821 (local address 127.0.0.1:37142)
0807 15:10:07.281432 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:07.281451 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:07.281550 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:07.282415 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:07.282420 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:07.282442 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:07.282546 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:07.282552 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:07.283660 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:07.283664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:07.284461 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:07.284471 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:07.285374 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:07.285377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:07.285471 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:07.285500 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 15:10:07.285546 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 15:10:07.285617 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 15:10:07.286458 26754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.281505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37142 (local address 127.15.226.66:44821)
0807 15:10:07.282173 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:07.282178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:07.282194 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:10:07.282259 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:07.282262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:07.282313 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:07.282405 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:07.282720 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:07.283527 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:07.284583 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:07.285232 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:07.286155 (+ 923us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:07.286244 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:07.286278 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 15:10:07.286341 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":87,"threads_started":1}
I20260807 15:10:07.287009 26319 tablet_service.cc:1467] Processing CreateTablet for tablet 076d2f0a7ab54936b1387f4c022df61c (DEFAULT_TABLE table=test-table [id=22bb727a8f5842d1a8a5558fd2a8191a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:07.287367 26319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 076d2f0a7ab54936b1387f4c022df61c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:07.287392 26758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.281928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54714 (local address 127.15.226.67:43089)
0807 15:10:07.282266 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:07.282273 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:07.282614 (+ 341us) server_negotiation.cc:407] Connection header received
0807 15:10:07.282895 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:07.282898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:07.282952 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:07.283041 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:07.283550 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:07.284356 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:07.285887 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:07.286485 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:07.286582 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:07.286644 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:07.287178 (+ 534us) server_negotiation.cc:299] Negotiation successful
0807 15:10:07.287271 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":100,"threads_started":1}
I20260807 15:10:07.287393 26755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.282061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:37605 (local address 127.0.0.1:53888)
0807 15:10:07.282878 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:07.282890 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:07.282973 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:07.283365 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:07.283368 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:07.283383 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:07.283503 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:07.283511 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:07.284498 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:07.284504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:07.285665 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:07.285675 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:07.286734 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:07.286738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:07.286984 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:07.287051 (+ 67us) client_negotiation.cc:769] Sending connection context
0807 15:10:07.287157 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 15:10:07.287255 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"mutex_wait_us":202,"thread_start_us":90,"threads_started":1}
I20260807 15:10:07.287469 26761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.282182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53888 (local address 127.15.226.65:37605)
0807 15:10:07.282422 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:07.282427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:07.283055 (+ 628us) server_negotiation.cc:407] Connection header received
0807 15:10:07.283127 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:07.283130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:07.283179 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:07.283257 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:07.283611 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:07.284376 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:07.285812 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:07.286457 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:07.286575 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:07.286644 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:07.287227 (+ 583us) server_negotiation.cc:299] Negotiation successful
0807 15:10:07.287324 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":102,"threads_started":1}
I20260807 15:10:07.288036 26134 tablet_service.cc:1467] Processing CreateTablet for tablet 076d2f0a7ab54936b1387f4c022df61c (DEFAULT_TABLE table=test-table [id=22bb727a8f5842d1a8a5558fd2a8191a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:07.288355 26134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 076d2f0a7ab54936b1387f4c022df61c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:07.288707 26760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.281805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:43089 (local address 127.0.0.1:54714)
0807 15:10:07.282505 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:07.282521 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:07.282610 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:07.283282 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:07.283285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:07.283305 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:07.283418 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:07.283425 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:07.284502 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:07.284505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:07.285756 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:07.285766 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:07.286739 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:07.286742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:07.286981 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:07.287053 (+ 72us) client_negotiation.cc:769] Sending connection context
0807 15:10:07.287604 (+ 551us) client_negotiation.cc:240] Negotiation successful
0807 15:10:07.287681 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"mutex_wait_us":143,"thread_start_us":117,"threads_started":1}
I20260807 15:10:07.289357 26526 tablet_service.cc:1467] Processing CreateTablet for tablet 076d2f0a7ab54936b1387f4c022df61c (DEFAULT_TABLE table=test-table [id=22bb727a8f5842d1a8a5558fd2a8191a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:07.289644 26526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 076d2f0a7ab54936b1387f4c022df61c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:07.290004 26762 tablet_bootstrap.cc:492] T 076d2f0a7ab54936b1387f4c022df61c P f448c8ea9dc5445496f57727ac1061bb: Bootstrap starting.
I20260807 15:10:07.291116 26762 tablet_bootstrap.cc:654] T 076d2f0a7ab54936b1387f4c022df61c P f448c8ea9dc5445496f57727ac1061bb: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:07.291329 26764 tablet_bootstrap.cc:492] T 076d2f0a7ab54936b1387f4c022df61c P 60340eaffc12444aaf7a470a0634f9ac: Bootstrap starting.
I20260807 15:10:07.292290 26764 tablet_bootstrap.cc:654] T 076d2f0a7ab54936b1387f4c022df61c P 60340eaffc12444aaf7a470a0634f9ac: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:07.292670 26766 tablet_bootstrap.cc:492] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65: Bootstrap starting.
I20260807 15:10:07.293767 26764 tablet_bootstrap.cc:492] T 076d2f0a7ab54936b1387f4c022df61c P 60340eaffc12444aaf7a470a0634f9ac: No bootstrap required, opened a new log
I20260807 15:10:07.293767 26762 tablet_bootstrap.cc:492] T 076d2f0a7ab54936b1387f4c022df61c P f448c8ea9dc5445496f57727ac1061bb: No bootstrap required, opened a new log
I20260807 15:10:07.293869 26764 ts_tablet_manager.cc:1397] T 076d2f0a7ab54936b1387f4c022df61c P 60340eaffc12444aaf7a470a0634f9ac: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:10:07.293869 26762 ts_tablet_manager.cc:1397] T 076d2f0a7ab54936b1387f4c022df61c P f448c8ea9dc5445496f57727ac1061bb: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 15:10:07.293924 26766 tablet_bootstrap.cc:654] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:07.294330 26764 raft_consensus.cc:348] T 076d2f0a7ab54936b1387f4c022df61c P 60340eaffc12444aaf7a470a0634f9ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f448c8ea9dc5445496f57727ac1061bb" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44821 } } peers { permanent_uuid: "60340eaffc12444aaf7a470a0634f9ac" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37605 } } peers { permanent_uuid: "b85d9bf8304749cea743efe6d297ea65" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43089 } }
I20260807 15:10:07.294330 26762 raft_consensus.cc:348] T 076d2f0a7ab54936b1387f4c022df61c P f448c8ea9dc5445496f57727ac1061bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f448c8ea9dc5445496f57727ac1061bb" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44821 } } peers { permanent_uuid: "60340eaffc12444aaf7a470a0634f9ac" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37605 } } peers { permanent_uuid: "b85d9bf8304749cea743efe6d297ea65" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43089 } }
I20260807 15:10:07.294463 26764 raft_consensus.cc:374] T 076d2f0a7ab54936b1387f4c022df61c P 60340eaffc12444aaf7a470a0634f9ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:07.294463 26762 raft_consensus.cc:374] T 076d2f0a7ab54936b1387f4c022df61c P f448c8ea9dc5445496f57727ac1061bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:07.294502 26764 raft_consensus.cc:729] T 076d2f0a7ab54936b1387f4c022df61c P 60340eaffc12444aaf7a470a0634f9ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60340eaffc12444aaf7a470a0634f9ac, State: Initialized, Role: FOLLOWER
I20260807 15:10:07.294500 26762 raft_consensus.cc:729] T 076d2f0a7ab54936b1387f4c022df61c P f448c8ea9dc5445496f57727ac1061bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f448c8ea9dc5445496f57727ac1061bb, State: Initialized, Role: FOLLOWER
I20260807 15:10:07.294637 26762 consensus_queue.cc:260] T 076d2f0a7ab54936b1387f4c022df61c P f448c8ea9dc5445496f57727ac1061bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f448c8ea9dc5445496f57727ac1061bb" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44821 } } peers { permanent_uuid: "60340eaffc12444aaf7a470a0634f9ac" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37605 } } peers { permanent_uuid: "b85d9bf8304749cea743efe6d297ea65" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43089 } }
I20260807 15:10:07.294637 26764 consensus_queue.cc:260] T 076d2f0a7ab54936b1387f4c022df61c P 60340eaffc12444aaf7a470a0634f9ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f448c8ea9dc5445496f57727ac1061bb" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44821 } } peers { permanent_uuid: "60340eaffc12444aaf7a470a0634f9ac" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37605 } } peers { permanent_uuid: "b85d9bf8304749cea743efe6d297ea65" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43089 } }
I20260807 15:10:07.295064 26764 ts_tablet_manager.cc:1428] T 076d2f0a7ab54936b1387f4c022df61c P 60340eaffc12444aaf7a470a0634f9ac: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:10:07.295064 26762 ts_tablet_manager.cc:1428] T 076d2f0a7ab54936b1387f4c022df61c P f448c8ea9dc5445496f57727ac1061bb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:10:07.295270 26766 tablet_bootstrap.cc:492] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65: No bootstrap required, opened a new log
I20260807 15:10:07.295353 26766 ts_tablet_manager.cc:1397] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:10:07.295779 26766 raft_consensus.cc:348] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f448c8ea9dc5445496f57727ac1061bb" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44821 } } peers { permanent_uuid: "60340eaffc12444aaf7a470a0634f9ac" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37605 } } peers { permanent_uuid: "b85d9bf8304749cea743efe6d297ea65" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43089 } }
I20260807 15:10:07.295909 26766 raft_consensus.cc:374] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:07.295946 26766 raft_consensus.cc:729] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b85d9bf8304749cea743efe6d297ea65, State: Initialized, Role: FOLLOWER
I20260807 15:10:07.296372 26732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.296103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.65:45339)
0807 15:10:07.296228 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:07.296296 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 15:10:07.296101 26766 consensus_queue.cc:260] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f448c8ea9dc5445496f57727ac1061bb" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44821 } } peers { permanent_uuid: "60340eaffc12444aaf7a470a0634f9ac" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37605 } } peers { permanent_uuid: "b85d9bf8304749cea743efe6d297ea65" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43089 } }
I20260807 15:10:07.296947 26766 ts_tablet_manager.cc:1428] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:10:07.298025 26733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.297458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.67:57417)
0807 15:10:07.297878 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:07.297946 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335}
I20260807 15:10:07.298080 26231 heartbeater.cc:503] Master 127.15.226.125:45651 was elected leader, sending a full tablet report...
I20260807 15:10:07.298873 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.298513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.66:42611)
0807 15:10:07.298720 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:07.298779 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260807 15:10:07.299736 26732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.299437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.65:50369)
0807 15:10:07.299606 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:07.299658 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260807 15:10:07.300504 26618 heartbeater.cc:503] Master 127.15.226.125:45651 was elected leader, sending a full tablet report...
I20260807 15:10:07.300508 26425 heartbeater.cc:503] Master 127.15.226.125:45651 was elected leader, sending a full tablet report...
I20260807 15:10:07.301208 26733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.298835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.67:47609)
0807 15:10:07.301051 (+ 2216us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:07.301112 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2137}
I20260807 15:10:07.301988 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.301658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.66:36989)
0807 15:10:07.301854 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:07.301910 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260807 15:10:07.342470 26772 raft_consensus.cc:482] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:10:07.342726 26772 raft_consensus.cc:504] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f448c8ea9dc5445496f57727ac1061bb" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44821 } } peers { permanent_uuid: "60340eaffc12444aaf7a470a0634f9ac" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37605 } } peers { permanent_uuid: "b85d9bf8304749cea743efe6d297ea65" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43089 } }
I20260807 15:10:07.343339 26772 leader_election.cc:288] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f448c8ea9dc5445496f57727ac1061bb (127.15.226.66:44821), 60340eaffc12444aaf7a470a0634f9ac (127.15.226.65:37605)
I20260807 15:10:07.343945 26349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "076d2f0a7ab54936b1387f4c022df61c" candidate_uuid: "b85d9bf8304749cea743efe6d297ea65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f448c8ea9dc5445496f57727ac1061bb" is_pre_election: true
I20260807 15:10:07.344025 26160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "076d2f0a7ab54936b1387f4c022df61c" candidate_uuid: "b85d9bf8304749cea743efe6d297ea65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60340eaffc12444aaf7a470a0634f9ac" is_pre_election: true
I20260807 15:10:07.344135 26349 raft_consensus.cc:2454] T 076d2f0a7ab54936b1387f4c022df61c P f448c8ea9dc5445496f57727ac1061bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b85d9bf8304749cea743efe6d297ea65 in term 0.
I20260807 15:10:07.344158 26160 raft_consensus.cc:2454] T 076d2f0a7ab54936b1387f4c022df61c P 60340eaffc12444aaf7a470a0634f9ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b85d9bf8304749cea743efe6d297ea65 in term 0.
I20260807 15:10:07.344583 26457 leader_election.cc:302] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60340eaffc12444aaf7a470a0634f9ac, b85d9bf8304749cea743efe6d297ea65; no voters:
I20260807 15:10:07.344812 26772 raft_consensus.cc:2781] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:10:07.344882 26772 raft_consensus.cc:482] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:10:07.344950 26772 raft_consensus.cc:3037] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:07.346199 26772 raft_consensus.cc:504] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f448c8ea9dc5445496f57727ac1061bb" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44821 } } peers { permanent_uuid: "60340eaffc12444aaf7a470a0634f9ac" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37605 } } peers { permanent_uuid: "b85d9bf8304749cea743efe6d297ea65" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43089 } }
I20260807 15:10:07.346648 26772 leader_election.cc:288] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65 [CANDIDATE]: Term 1 election: Requested vote from peers f448c8ea9dc5445496f57727ac1061bb (127.15.226.66:44821), 60340eaffc12444aaf7a470a0634f9ac (127.15.226.65:37605)
I20260807 15:10:07.347133 26160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "076d2f0a7ab54936b1387f4c022df61c" candidate_uuid: "b85d9bf8304749cea743efe6d297ea65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60340eaffc12444aaf7a470a0634f9ac"
I20260807 15:10:07.347175 26349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "076d2f0a7ab54936b1387f4c022df61c" candidate_uuid: "b85d9bf8304749cea743efe6d297ea65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f448c8ea9dc5445496f57727ac1061bb"
I20260807 15:10:07.347298 26160 raft_consensus.cc:3037] T 076d2f0a7ab54936b1387f4c022df61c P 60340eaffc12444aaf7a470a0634f9ac [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:07.347330 26349 raft_consensus.cc:3037] T 076d2f0a7ab54936b1387f4c022df61c P f448c8ea9dc5445496f57727ac1061bb [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:07.348356 26349 raft_consensus.cc:2454] T 076d2f0a7ab54936b1387f4c022df61c P f448c8ea9dc5445496f57727ac1061bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b85d9bf8304749cea743efe6d297ea65 in term 1.
I20260807 15:10:07.348366 26160 raft_consensus.cc:2454] T 076d2f0a7ab54936b1387f4c022df61c P 60340eaffc12444aaf7a470a0634f9ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b85d9bf8304749cea743efe6d297ea65 in term 1.
I20260807 15:10:07.348786 26457 leader_election.cc:302] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60340eaffc12444aaf7a470a0634f9ac, b85d9bf8304749cea743efe6d297ea65; no voters:
I20260807 15:10:07.349058 26772 raft_consensus.cc:2781] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:07.349340 26772 raft_consensus.cc:686] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65 [term 1 LEADER]: Becoming Leader. State: Replica: b85d9bf8304749cea743efe6d297ea65, State: Running, Role: LEADER
I20260807 15:10:07.349551 26772 consensus_queue.cc:237] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f448c8ea9dc5445496f57727ac1061bb" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44821 } } peers { permanent_uuid: "60340eaffc12444aaf7a470a0634f9ac" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37605 } } peers { permanent_uuid: "b85d9bf8304749cea743efe6d297ea65" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43089 } }
I20260807 15:10:07.350292 26733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.349967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.67:42117)
0807 15:10:07.350132 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:07.350203 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 15:10:07.351605 25706 catalog_manager.cc:5495] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65 reported cstate change: term changed from 0 to 1, leader changed from <none> to b85d9bf8304749cea743efe6d297ea65 (127.15.226.67). New cstate: current_term: 1 leader_uuid: "b85d9bf8304749cea743efe6d297ea65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f448c8ea9dc5445496f57727ac1061bb" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60340eaffc12444aaf7a470a0634f9ac" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b85d9bf8304749cea743efe6d297ea65" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43089 } health_report { overall_health: HEALTHY } } }
I20260807 15:10:07.376876 26758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.373814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54720 (local address 127.15.226.67:43089)
0807 15:10:07.373976 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:07.373979 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:07.373994 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:10:07.374044 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:07.374046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:07.374092 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:07.374163 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:10:07.374410 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:07.374963 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:07.375801 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:07.376014 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:07.376091 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:07.376329 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:10:07.376416 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:10:07.376484 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:07.376604 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 15:10:07.376658 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 15:10:07.379526 26349 raft_consensus.cc:1261] T 076d2f0a7ab54936b1387f4c022df61c P f448c8ea9dc5445496f57727ac1061bb [term 1 FOLLOWER]: Refusing update from remote peer b85d9bf8304749cea743efe6d297ea65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:07.379489 26160 raft_consensus.cc:1261] T 076d2f0a7ab54936b1387f4c022df61c P 60340eaffc12444aaf7a470a0634f9ac [term 1 FOLLOWER]: Refusing update from remote peer b85d9bf8304749cea743efe6d297ea65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:07.380081 26772 consensus_queue.cc:1035] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60340eaffc12444aaf7a470a0634f9ac" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 37605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:07.380249 26773 consensus_queue.cc:1035] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f448c8ea9dc5445496f57727ac1061bb" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:07.383046 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.382709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.66:43489)
0807 15:10:07.382886 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:07.382954 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260807 15:10:07.383342 26732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.383009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.65:54585)
0807 15:10:07.383187 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:07.383244 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260807 15:10:07.386966 26733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.386485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.67:34567)
0807 15:10:07.386807 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:07.386875 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229}
I20260807 15:10:07.387138 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.386771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.66:49173)
0807 15:10:07.386999 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:07.387058 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
I20260807 15:10:07.387907 26732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.387445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.65:36191)
0807 15:10:07.387745 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:07.387810 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228}
I20260807 15:10:07.391474 26733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.391179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44957 (local address 127.15.226.67:38453)
0807 15:10:07.391317 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:07.391391 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:44957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 15:10:07.393972 25706 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:07.394134 25706 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:07.398388 25706 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 582deee640a1481cbf7a09c88c6269f7: Sending DeleteTablet for 3 replicas of tablet 076d2f0a7ab54936b1387f4c022df61c
I20260807 15:10:07.399259 26319 tablet_service.cc:1514] Processing DeleteTablet for tablet 076d2f0a7ab54936b1387f4c022df61c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:07 UTC) from {username='oryx', principal='oryx/127.15.226.125@KRBTEST.COM'} at 127.0.0.1:37142
I20260807 15:10:07.399296 26134 tablet_service.cc:1514] Processing DeleteTablet for tablet 076d2f0a7ab54936b1387f4c022df61c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:07 UTC) from {username='oryx', principal='oryx/127.15.226.125@KRBTEST.COM'} at 127.0.0.1:53888
I20260807 15:10:07.399401 26526 tablet_service.cc:1514] Processing DeleteTablet for tablet 076d2f0a7ab54936b1387f4c022df61c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:07 UTC) from {username='oryx', principal='oryx/127.15.226.125@KRBTEST.COM'} at 127.0.0.1:54714
I20260807 15:10:07.399834 26808 tablet_replica.cc:330] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65: stopping tablet replica
I20260807 15:10:07.399973 26796 tablet_replica.cc:330] T 076d2f0a7ab54936b1387f4c022df61c P f448c8ea9dc5445496f57727ac1061bb: stopping tablet replica
I20260807 15:10:07.400032 26808 raft_consensus.cc:2229] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:10:07.400282 26808 raft_consensus.cc:2258] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:07.400316 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 25919
I20260807 15:10:07.409317 26796 raft_consensus.cc:2229] T 076d2f0a7ab54936b1387f4c022df61c P f448c8ea9dc5445496f57727ac1061bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:10:07.409612 26796 raft_consensus.cc:2258] T 076d2f0a7ab54936b1387f4c022df61c P f448c8ea9dc5445496f57727ac1061bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:07.410012 26808 ts_tablet_manager.cc:1905] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:07.410454 26796 ts_tablet_manager.cc:1905] T 076d2f0a7ab54936b1387f4c022df61c P f448c8ea9dc5445496f57727ac1061bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:07.413290 26808 ts_tablet_manager.cc:1918] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:07.413371 26808 log.cc:1196] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/076d2f0a7ab54936b1387f4c022df61c
I20260807 15:10:07.413612 26808 ts_tablet_manager.cc:1939] T 076d2f0a7ab54936b1387f4c022df61c P b85d9bf8304749cea743efe6d297ea65: Deleting consensus metadata
I20260807 15:10:07.414253 26796 ts_tablet_manager.cc:1918] T 076d2f0a7ab54936b1387f4c022df61c P f448c8ea9dc5445496f57727ac1061bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:07.414340 26796 log.cc:1196] T 076d2f0a7ab54936b1387f4c022df61c P f448c8ea9dc5445496f57727ac1061bb: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/076d2f0a7ab54936b1387f4c022df61c
I20260807 15:10:07.414428 25690 catalog_manager.cc:4841] TS b85d9bf8304749cea743efe6d297ea65 (127.15.226.67:43089): tablet 076d2f0a7ab54936b1387f4c022df61c (table test-table [id=22bb727a8f5842d1a8a5558fd2a8191a]) successfully deleted
I20260807 15:10:07.414677 26796 ts_tablet_manager.cc:1939] T 076d2f0a7ab54936b1387f4c022df61c P f448c8ea9dc5445496f57727ac1061bb: Deleting consensus metadata
I20260807 15:10:07.415498 25689 catalog_manager.cc:4841] TS f448c8ea9dc5445496f57727ac1061bb (127.15.226.66:44821): tablet 076d2f0a7ab54936b1387f4c022df61c (table test-table [id=22bb727a8f5842d1a8a5558fd2a8191a]) successfully deleted
I20260807 15:10:07.421869 26760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:07.421586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:37605 (local address 127.0.0.1:53904)
0807 15:10:07.421725 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:07.421780 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.65:37605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 15:10:07.421876 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 26240
W20260807 15:10:07.422123 25694 catalog_manager.cc:4584] TS 60340eaffc12444aaf7a470a0634f9ac (127.15.226.65:37605): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 076d2f0a7ab54936b1387f4c022df61c: Network error: Client connection negotiation failed: client connection to 127.15.226.65:37605: connect: Connection refused (error 111)
I20260807 15:10:07.435990 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 26430
I20260807 15:10:07.455555 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 25521
I20260807 15:10:07.467103 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 25658
2026-08-07T15:10:07Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7860 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:10:07 dist-test-slave-3kk6 krb5kdc[26814](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:10:07 dist-test-slave-3kk6 krb5kdc[26814](info): set up 2 sockets
Aug 07 15:10:07 dist-test-slave-3kk6 krb5kdc[26814](info): commencing operation
krb5kdc: starting...
W20260807 15:10:08.725883 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:10:08 dist-test-slave-3kk6 krb5kdc[26814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:10:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:08Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.126.keytab.
Entry for principal oryx/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.126.keytab.
I20260807 15:10:08.866994 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:41059
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.126.keytab
--principal=oryx/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:32887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:41059
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:08.987062 26833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:08.987447 26833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:08.987534 26833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:08.991813 26833 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:08.991945 26833 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 15:10:08.991977 26833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:08.992023 26833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:08.992084 26833 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:08.992141 26833 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:08.997859 26833 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:32887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:41059
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.126.keytab
--principal=oryx/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:41059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:08.999128 26833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:09.000264 26833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:09.009809 26845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:09.009933 26833 server_base.cc:1034] running on GCE node
W20260807 15:10:09.009891 26841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:09.009809 26839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:09.010867 26833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:09.012055 26833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:09.013231 26833 hybrid_clock.cc:648] HybridClock initialized: now 1786115409013217 us; error 35 us; skew 500 ppm
Aug 07 15:10:09 dist-test-slave-3kk6 krb5kdc[26814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115409, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:09.016523 26833 init.cc:377] Logged in from keytab as oryx/127.15.226.126@KRBTEST.COM (short username oryx)
I20260807 15:10:09.017535 26833 webserver.cc:460] Webserver started at http://127.15.226.126:39521/ using document root <none> and password file <none>
I20260807 15:10:09.018142 26833 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:09.018211 26833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:09.018478 26833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:09.020816 26833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "742bd21cffe34aef89250a542ac1e8fa"
format_stamp: "Formatted at 2026-08-07 15:10:09 on dist-test-slave-3kk6"
server_key: "a808bf4058fed0d1a324679db803f3ce"
server_key_iv: "470ff17548bedb980c9c7cf94fe34fb9"
server_key_version: "encryptionkey@0"
I20260807 15:10:09.021404 26833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "742bd21cffe34aef89250a542ac1e8fa"
format_stamp: "Formatted at 2026-08-07 15:10:09 on dist-test-slave-3kk6"
server_key: "a808bf4058fed0d1a324679db803f3ce"
server_key_iv: "470ff17548bedb980c9c7cf94fe34fb9"
server_key_version: "encryptionkey@0"
I20260807 15:10:09.025585 26833 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:10:09.028205 26859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:09.029554 26833 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:10:09.029729 26833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "742bd21cffe34aef89250a542ac1e8fa"
format_stamp: "Formatted at 2026-08-07 15:10:09 on dist-test-slave-3kk6"
server_key: "a808bf4058fed0d1a324679db803f3ce"
server_key_iv: "470ff17548bedb980c9c7cf94fe34fb9"
server_key_version: "encryptionkey@0"
I20260807 15:10:09.029908 26833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:09.040958 26833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:09.044634 26833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:09.044889 26833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:09.053891 26833 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:41059
I20260807 15:10:09.053906 26954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:41059 every 8 connection(s)
I20260807 15:10:09.055411 26833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:09.058374 26955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:09.064486 26955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa: Bootstrap starting.
I20260807 15:10:09.065624 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 26833
I20260807 15:10:09.065804 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:10:09.066149 16265 external_mini_cluster.cc:1428] Setting key 8222956a72d4fafb890e4db79229d9e4
I20260807 15:10:09.067807 26955 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:09.068640 26955 log.cc:824] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:09.071019 26955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa: No bootstrap required, opened a new log
I20260807 15:10:09.075070 26955 raft_consensus.cc:348] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "742bd21cffe34aef89250a542ac1e8fa" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 41059 } }
I20260807 15:10:09.075387 26955 raft_consensus.cc:374] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:09.075485 26955 raft_consensus.cc:729] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 742bd21cffe34aef89250a542ac1e8fa, State: Initialized, Role: FOLLOWER
Aug 07 15:10:09 dist-test-slave-3kk6 krb5kdc[26814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM
I20260807 15:10:09.076013 26955 consensus_queue.cc:260] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "742bd21cffe34aef89250a542ac1e8fa" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 41059 } }
I20260807 15:10:09.076210 26955 raft_consensus.cc:388] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:09.076310 26955 raft_consensus.cc:482] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:09.076436 26955 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:09.077659 26955 raft_consensus.cc:504] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "742bd21cffe34aef89250a542ac1e8fa" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 41059 } }
I20260807 15:10:09.078081 26955 leader_election.cc:302] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 742bd21cffe34aef89250a542ac1e8fa; no voters:
I20260807 15:10:09.078477 26955 leader_election.cc:288] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:10:09.078563 26962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:09.078835 26962 raft_consensus.cc:686] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa [term 1 LEADER]: Becoming Leader. State: Replica: 742bd21cffe34aef89250a542ac1e8fa, State: Running, Role: LEADER
I20260807 15:10:09.079212 26962 consensus_queue.cc:237] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "742bd21cffe34aef89250a542ac1e8fa" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 41059 } }
I20260807 15:10:09.079954 26955 sys_catalog.cc:564] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:09.080564 26962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 742bd21cffe34aef89250a542ac1e8fa. Latest consensus state: current_term: 1 leader_uuid: "742bd21cffe34aef89250a542ac1e8fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "742bd21cffe34aef89250a542ac1e8fa" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 41059 } } }
I20260807 15:10:09.080801 26962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:09.081018 26963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "742bd21cffe34aef89250a542ac1e8fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "742bd21cffe34aef89250a542ac1e8fa" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 41059 } } }
I20260807 15:10:09.081151 26963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:09.081396 26978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:09.083427 26958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:09.067505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55952 (local address 127.15.226.126:41059)
0807 15:10:09.068087 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:09.068104 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:09.068150 (+ 46us) server_negotiation.cc:407] Connection header received
0807 15:10:09.068989 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:09.069011 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:09.069486 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:09.069932 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:09.070886 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.072007 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.072941 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.073465 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.073656 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:09.076807 (+ 3151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:09.076813 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:09.076828 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:09.076861 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:09.079034 (+ 2173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:09.079584 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:09.079590 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:09.079597 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:09.079736 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:09.080111 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:09.080114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:09.080118 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:09.080679 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:09.081765 (+ 1086us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:09.081946 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 15:10:09.082277 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":178,"threads_started":1}
I20260807 15:10:09.084882 26978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:10:09.091308 26978 catalog_manager.cc:1347] Generated new cluster ID: b0b7c96730d64649a9b96bcffdde5b47
I20260807 15:10:09.091414 26978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:10:09.121402 26978 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:10:09.122449 26978 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:09.131685 26978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa: Generated new TSK 0
I20260807 15:10:09.132543 26978 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.226.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.226.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.226.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.65.keytab.
Entry for principal oryx/127.15.226.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.65.keytab.
I20260807 15:10:09.174947 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:41059
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.65.keytab
--principal=oryx/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:32887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:09.300336 26998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:09.300657 26998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:09.300733 26998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:09.305092 26998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:09.305182 26998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:09.305225 26998 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:10:09.305290 26998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:10:09.311007 26998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:32887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.65.keytab
--principal=oryx/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.226.126:41059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:09.312286 26998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:09.313422 26998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:09.324275 27006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:09.324304 27009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:09.324275 27007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:09.325084 26998 server_base.cc:1034] running on GCE node
I20260807 15:10:09.325626 26998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:09.326354 26998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:09.327559 26998 hybrid_clock.cc:648] HybridClock initialized: now 1786115409327533 us; error 54 us; skew 500 ppm
Aug 07 15:10:09 dist-test-slave-3kk6 krb5kdc[26814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115409, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:09.330843 26998 init.cc:377] Logged in from keytab as oryx/127.15.226.65@KRBTEST.COM (short username oryx)
I20260807 15:10:09.331952 26998 webserver.cc:460] Webserver started at http://127.15.226.65:43413/ using document root <none> and password file <none>
I20260807 15:10:09.332582 26998 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:09.332685 26998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:09.332948 26998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:09.335532 26998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "0dc5606586144eb182d4d1459833528c"
format_stamp: "Formatted at 2026-08-07 15:10:09 on dist-test-slave-3kk6"
server_key: "e991cf905d49c6825d70f5806ea85b92"
server_key_iv: "2705e8c7bfdba66bb1204be0a1c97226"
server_key_version: "encryptionkey@0"
I20260807 15:10:09.336170 26998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "0dc5606586144eb182d4d1459833528c"
format_stamp: "Formatted at 2026-08-07 15:10:09 on dist-test-slave-3kk6"
server_key: "e991cf905d49c6825d70f5806ea85b92"
server_key_iv: "2705e8c7bfdba66bb1204be0a1c97226"
server_key_version: "encryptionkey@0"
I20260807 15:10:09.340503 26998 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:10:09.343364 27017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:09.344648 26998 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:10:09.344820 26998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "0dc5606586144eb182d4d1459833528c"
format_stamp: "Formatted at 2026-08-07 15:10:09 on dist-test-slave-3kk6"
server_key: "e991cf905d49c6825d70f5806ea85b92"
server_key_iv: "2705e8c7bfdba66bb1204be0a1c97226"
server_key_version: "encryptionkey@0"
I20260807 15:10:09.344964 26998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:09.372273 26998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:09.376119 26998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:09.376366 26998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:09.378005 26998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:09.378087 26998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:09.378166 26998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:09.378211 26998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:10:09 dist-test-slave-3kk6 krb5kdc[26814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115409, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM
I20260807 15:10:09.390618 26998 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:38203
I20260807 15:10:09.390733 27184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:38203 every 8 connection(s)
I20260807 15:10:09.391996 26998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:10:09.394274 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 26998
I20260807 15:10:09.394430 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:10:09.394824 26958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:09.379155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:37189 (local address 127.15.226.126:41059)
0807 15:10:09.379292 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:09.379296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:09.380488 (+ 1192us) server_negotiation.cc:407] Connection header received
0807 15:10:09.381946 (+ 1458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:09.381950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:09.382015 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:09.382118 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:09.384047 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.384962 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.386365 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.386625 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.386795 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:09.390431 (+ 3636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:09.390443 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:09.390445 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:09.390475 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:09.392206 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:09.392893 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:09.392896 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:09.392898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:09.392998 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:09.393414 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:09.393417 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:09.393418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:09.393630 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:09.393724 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:09.394458 (+ 734us) server_negotiation.cc:299] Negotiation successful
0807 15:10:09.394613 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 15:10:09.394827 16265 external_mini_cluster.cc:1428] Setting key c3bbe5ba7763eca8775adfaa448271b8
I20260807 15:10:09.395961 27032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:09.379739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:41059 (local address 127.15.226.65:37189)
0807 15:10:09.380298 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:09.380348 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:09.381695 (+ 1347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:09.382374 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:09.382384 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:09.382981 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:09.383774 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:09.383798 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.385149 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.385152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:09.386229 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:09.386239 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.386860 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.386864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:09.387056 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:09.388186 (+ 1130us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:09.388212 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:09.390178 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:09.392359 (+ 2181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:09.392365 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:09.392381 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:09.392779 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:09.393115 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:09.393118 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:09.393119 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:09.393294 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:09.393778 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:09.393784 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:09.394108 (+ 324us) client_negotiation.cc:769] Sending connection context
0807 15:10:09.394371 (+ 263us) client_negotiation.cc:240] Negotiation successful
0807 15:10:09.394869 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":146,"threads_started":1}
I20260807 15:10:09.397392 27185 heartbeater.cc:347] Connected to a master server at 127.15.226.126:41059
I20260807 15:10:09.397589 27185 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:09.398144 27185 heartbeater.cc:511] Master 127.15.226.126:41059 requested a full tablet report, sending...
I20260807 15:10:09.400076 26884 ts_manager.cc:194] Registered new tserver with Master: 0dc5606586144eb182d4d1459833528c (127.15.226.65:38203)
I20260807 15:10:09.401593 26884 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.226.65@KRBTEST.COM'} at 127.15.226.65:37189
WARNING: no policy specified for oryx/127.15.226.66@KRBTEST.COM; defaulting to no policy
I20260807 15:10:09.408814 26958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:09.400314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:34485 (local address 127.15.226.126:41059)
0807 15:10:09.400439 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:09.400443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:09.401026 (+ 583us) server_negotiation.cc:407] Connection header received
0807 15:10:09.401148 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:09.401151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:09.401197 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:09.401283 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:09.402000 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.402751 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.403658 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.403907 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.404023 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:09.405274 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:09.405278 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:09.405281 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:09.405307 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:09.406996 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:09.407470 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:09.407474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:09.407476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:09.407570 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:09.407882 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:09.407885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:09.407887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:09.408095 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:09.408213 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:09.408503 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 15:10:09.408631 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 15:10:09.408833 27192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:09.400537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:41059 (local address 127.15.226.65:34485)
0807 15:10:09.400911 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:09.400931 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:09.401047 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:09.401327 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:09.401333 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:09.401604 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:09.401837 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:09.401851 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.402897 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.402900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:09.403527 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:09.403536 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.404045 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.404050 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:09.404154 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:09.404737 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:09.404749 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:09.405121 (+ 372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:09.407121 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:09.407125 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:09.407127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:09.407351 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:09.407671 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:09.407674 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:09.407676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:09.407775 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:09.408204 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:09.408207 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:09.408319 (+ 112us) client_negotiation.cc:769] Sending connection context
0807 15:10:09.408439 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 15:10:09.408623 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":2,"thread_start_us":129,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.226.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.226.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.66.keytab.
Entry for principal oryx/127.15.226.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.66.keytab.
I20260807 15:10:09.430310 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:41059
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.66.keytab
--principal=oryx/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:32887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:09.553531 27195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:09.553871 27195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:09.553920 27195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:09.558243 27195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:09.558326 27195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:09.558370 27195 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:10:09.558406 27195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:10:09.563974 27195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:32887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.66.keytab
--principal=oryx/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.226.126:41059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:09.565137 27195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:09.566188 27195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:09.577067 27205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:09.577082 27207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:09.577082 27204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:09.577520 27195 server_base.cc:1034] running on GCE node
I20260807 15:10:09.577968 27195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:09.578807 27195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:09.580024 27195 hybrid_clock.cc:648] HybridClock initialized: now 1786115409579993 us; error 55 us; skew 500 ppm
Aug 07 15:10:09 dist-test-slave-3kk6 krb5kdc[26814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115409, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:09.583464 27195 init.cc:377] Logged in from keytab as oryx/127.15.226.66@KRBTEST.COM (short username oryx)
I20260807 15:10:09.584470 27195 webserver.cc:460] Webserver started at http://127.15.226.66:36799/ using document root <none> and password file <none>
I20260807 15:10:09.585084 27195 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:09.585156 27195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:09.585455 27195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:09.587878 27195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "b61995e07c35456694b60e70309f2eef"
format_stamp: "Formatted at 2026-08-07 15:10:09 on dist-test-slave-3kk6"
server_key: "6257c0906f372a4083e0ce0d2c7975ce"
server_key_iv: "9e6787f8e07139cc809ee84cc404c485"
server_key_version: "encryptionkey@0"
I20260807 15:10:09.588615 27195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "b61995e07c35456694b60e70309f2eef"
format_stamp: "Formatted at 2026-08-07 15:10:09 on dist-test-slave-3kk6"
server_key: "6257c0906f372a4083e0ce0d2c7975ce"
server_key_iv: "9e6787f8e07139cc809ee84cc404c485"
server_key_version: "encryptionkey@0"
I20260807 15:10:09.593003 27195 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:10:09.595888 27217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:09.597266 27195 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:10:09.597448 27195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "b61995e07c35456694b60e70309f2eef"
format_stamp: "Formatted at 2026-08-07 15:10:09 on dist-test-slave-3kk6"
server_key: "6257c0906f372a4083e0ce0d2c7975ce"
server_key_iv: "9e6787f8e07139cc809ee84cc404c485"
server_key_version: "encryptionkey@0"
I20260807 15:10:09.597566 27195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:09.613621 27195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:09.617538 27195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:09.617787 27195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:09.619659 27195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:09.619742 27195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:09.619843 27195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:09.619887 27195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:10:09 dist-test-slave-3kk6 krb5kdc[26814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115409, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM
I20260807 15:10:09.631866 27195 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:42627
I20260807 15:10:09.631987 27381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:42627 every 8 connection(s)
I20260807 15:10:09.633236 27195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:10:09.636539 26958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:09.620618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:33861 (local address 127.15.226.126:41059)
0807 15:10:09.620812 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:09.620819 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:09.621745 (+ 926us) server_negotiation.cc:407] Connection header received
0807 15:10:09.622925 (+ 1180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:09.622930 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:09.623004 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:09.623073 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:09.625096 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.626017 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.628167 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.628524 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.628686 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:09.632296 (+ 3610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:09.632301 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:09.632304 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:09.632339 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:09.634199 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:09.635013 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:09.635017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:09.635019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:09.635120 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:09.635458 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:09.635460 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:09.635461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:09.635655 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:09.635733 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:09.636295 (+ 562us) server_negotiation.cc:299] Negotiation successful
0807 15:10:09.636416 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 15:10:09.637476 27236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:09.621036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:41059 (local address 127.15.226.66:33861)
0807 15:10:09.621550 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:09.621606 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:09.622610 (+ 1004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:09.623620 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:09.623628 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:09.624118 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:09.624916 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:09.624941 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.626216 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.626223 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:09.627957 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:09.627975 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.628798 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.628803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:09.628977 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:09.629862 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:09.629887 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:09.632070 (+ 2183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:09.634368 (+ 2298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:09.634374 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:09.634398 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:09.634844 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:09.635213 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:09.635215 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:09.635216 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:09.635368 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:09.635790 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:09.635795 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:09.636052 (+ 257us) client_negotiation.cc:769] Sending connection context
0807 15:10:09.636212 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 15:10:09.636501 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":152,"threads_started":1}
I20260807 15:10:09.638926 27384 heartbeater.cc:347] Connected to a master server at 127.15.226.126:41059
I20260807 15:10:09.639127 27384 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:09.639624 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 27195
I20260807 15:10:09.639751 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:10:09.639870 27384 heartbeater.cc:511] Master 127.15.226.126:41059 requested a full tablet report, sending...
I20260807 15:10:09.640101 16265 external_mini_cluster.cc:1428] Setting key 487deaba451d006aa9cae42706535fe4
I20260807 15:10:09.641152 26884 ts_manager.cc:194] Registered new tserver with Master: b61995e07c35456694b60e70309f2eef (127.15.226.66:42627)
I20260807 15:10:09.641916 26884 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.226.66@KRBTEST.COM'} at 127.15.226.66:33861
I20260807 15:10:09.650319 26958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:09.641906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:34467 (local address 127.15.226.126:41059)
0807 15:10:09.642035 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:09.642039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:09.642196 (+ 157us) server_negotiation.cc:407] Connection header received
0807 15:10:09.642389 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:09.642392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:09.642438 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:09.642526 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:09.643086 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.643702 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.644728 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.644948 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.645039 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:09.646373 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:09.646376 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:09.646378 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:09.646404 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:09.648174 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:09.648823 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:09.648826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:09.648827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:09.648913 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:09.649278 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:09.649301 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:09.649305 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:09.649582 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:09.649702 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:09.650023 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 15:10:09.650141 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:10:09.650364 27392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:09.641528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:41059 (local address 127.15.226.66:34467)
0807 15:10:09.642101 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:09.642117 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:09.642268 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:09.642544 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:09.642547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:09.642821 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:09.642966 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:09.642973 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.643832 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.643835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:09.644558 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:09.644570 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.645075 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.645081 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:09.645201 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:09.645822 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:09.645843 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:09.646222 (+ 379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:09.648348 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:09.648362 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:09.648365 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:09.648631 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:09.649031 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:09.649036 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:09.649038 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:09.649139 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:09.649692 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:09.649696 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:09.649843 (+ 147us) client_negotiation.cc:769] Sending connection context
0807 15:10:09.649959 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 15:10:09.650141 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"mutex_wait_us":23,"thread_start_us":169,"threads_started":1}
WARNING: no policy specified for oryx/127.15.226.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.226.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.226.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.67.keytab.
Entry for principal oryx/127.15.226.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.67.keytab.
I20260807 15:10:09.673933 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:41059
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.67.keytab
--principal=oryx/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:32887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:09.794864 27396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:09.795215 27396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:09.795310 27396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:09.799713 27396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:09.799817 27396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:09.799917 27396 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 15:10:09.800024 27396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:10:09.806028 27396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:32887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.67.keytab
--principal=oryx/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.226.126:41059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:09.807430 27396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:09.808562 27396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:09.819013 27406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:09.819108 27402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:09.819065 27403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:09.819480 27396 server_base.cc:1034] running on GCE node
I20260807 15:10:09.819937 27396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:09.820724 27396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:09.821923 27396 hybrid_clock.cc:648] HybridClock initialized: now 1786115409821888 us; error 56 us; skew 500 ppm
Aug 07 15:10:09 dist-test-slave-3kk6 krb5kdc[26814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115409, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:09.824945 27396 init.cc:377] Logged in from keytab as oryx/127.15.226.67@KRBTEST.COM (short username oryx)
I20260807 15:10:09.825956 27396 webserver.cc:460] Webserver started at http://127.15.226.67:46103/ using document root <none> and password file <none>
I20260807 15:10:09.826567 27396 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:09.826697 27396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:09.826959 27396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:09.829294 27396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "17aafb9028354f5ea9c4302d7d3c3dbb"
format_stamp: "Formatted at 2026-08-07 15:10:09 on dist-test-slave-3kk6"
server_key: "7b5b67c62befbd3f9fe20afdaef79521"
server_key_iv: "417a5d81f81b5fe7a15977afaa48a5a4"
server_key_version: "encryptionkey@0"
I20260807 15:10:09.829941 27396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "17aafb9028354f5ea9c4302d7d3c3dbb"
format_stamp: "Formatted at 2026-08-07 15:10:09 on dist-test-slave-3kk6"
server_key: "7b5b67c62befbd3f9fe20afdaef79521"
server_key_iv: "417a5d81f81b5fe7a15977afaa48a5a4"
server_key_version: "encryptionkey@0"
I20260807 15:10:09.834383 27396 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260807 15:10:09.837247 27417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:09.838711 27396 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:10:09.838910 27396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "17aafb9028354f5ea9c4302d7d3c3dbb"
format_stamp: "Formatted at 2026-08-07 15:10:09 on dist-test-slave-3kk6"
server_key: "7b5b67c62befbd3f9fe20afdaef79521"
server_key_iv: "417a5d81f81b5fe7a15977afaa48a5a4"
server_key_version: "encryptionkey@0"
I20260807 15:10:09.839061 27396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:09.851565 27396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:09.855067 27396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:09.855316 27396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:09.856992 27396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:09.857071 27396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:09.857143 27396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:09.857188 27396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 15:10:09 dist-test-slave-3kk6 krb5kdc[26814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115409, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM
I20260807 15:10:09.869923 27396 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:37875
I20260807 15:10:09.870033 27580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:37875 every 8 connection(s)
I20260807 15:10:09.871248 27396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:10:09.872071 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 27396
I20260807 15:10:09.872213 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:10:09.872577 16265 external_mini_cluster.cc:1428] Setting key 51714dec01c59715b5c820d784ddbf0b
I20260807 15:10:09.872635 26958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:09.857950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:55723 (local address 127.15.226.126:41059)
0807 15:10:09.858092 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:09.858095 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:09.859413 (+ 1318us) server_negotiation.cc:407] Connection header received
0807 15:10:09.860910 (+ 1497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:09.860916 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:09.860996 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:09.861101 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:09.862569 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.863340 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.864559 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.864877 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.864999 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:09.868058 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:09.868064 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:09.868072 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:09.868101 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:09.870009 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:09.870706 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:09.870711 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:09.870712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:09.870827 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:09.871205 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:09.871208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:09.871209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:09.871487 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:09.871581 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:09.872301 (+ 720us) server_negotiation.cc:299] Negotiation successful
0807 15:10:09.872458 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:10:09.873703 27432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:09.858510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:41059 (local address 127.15.226.67:55723)
0807 15:10:09.859178 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:09.859271 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:09.860598 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:09.861329 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:09.861337 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:09.861801 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:09.862396 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:09.862411 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.863467 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.863470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:09.864407 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:09.864418 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.865062 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.865066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:09.865286 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:09.866079 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:09.866120 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:09.867847 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:09.870158 (+ 2311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:09.870164 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:09.870178 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:09.870527 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:09.870917 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:09.870920 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:09.870922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:09.871090 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:09.871612 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:09.871621 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:09.872008 (+ 387us) client_negotiation.cc:769] Sending connection context
0807 15:10:09.872252 (+ 244us) client_negotiation.cc:240] Negotiation successful
0807 15:10:09.872604 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":147,"threads_started":1}
I20260807 15:10:09.875047 27581 heartbeater.cc:347] Connected to a master server at 127.15.226.126:41059
I20260807 15:10:09.875236 27581 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:09.875850 27581 heartbeater.cc:511] Master 127.15.226.126:41059 requested a full tablet report, sending...
I20260807 15:10:09.876952 26884 ts_manager.cc:194] Registered new tserver with Master: 17aafb9028354f5ea9c4302d7d3c3dbb (127.15.226.67:37875)
I20260807 15:10:09.877684 26884 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.226.67@KRBTEST.COM'} at 127.15.226.67:55723
I20260807 15:10:09.885082 26958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:09.877603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:58503 (local address 127.15.226.126:41059)
0807 15:10:09.877732 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:09.877737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:09.877968 (+ 231us) server_negotiation.cc:407] Connection header received
0807 15:10:09.878107 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:09.878109 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:09.878155 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:09.878244 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:09.878774 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.879277 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.880207 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.880434 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.880552 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:09.881920 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:09.881924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:09.881928 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:09.881978 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:09.883341 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:09.883795 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:09.883800 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:09.883801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:09.883891 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:09.884197 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:09.884200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:09.884201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:09.884402 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:09.884518 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:09.884795 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 15:10:09.884913 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:10:09.885098 27592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:09.877481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:41059 (local address 127.15.226.67:58503)
0807 15:10:09.877888 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:09.877907 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:09.878015 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:09.878263 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:09.878266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:09.878495 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:09.878620 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:09.878627 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.879428 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.879431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:09.880084 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:09.880095 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.880558 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.880561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:09.880654 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:09.881376 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:09.881398 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:09.881740 (+ 342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:09.883471 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:09.883474 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:09.883477 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:09.883690 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:09.883993 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:09.883996 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:09.883998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:09.884091 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:09.884508 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:09.884511 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:09.884617 (+ 106us) client_negotiation.cc:769] Sending connection context
0807 15:10:09.884732 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 15:10:09.884906 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":1,"thread_start_us":130,"threads_started":1}
I20260807 15:10:09.887091 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 15:10:09 dist-test-slave-3kk6 krb5kdc[26814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115409, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 15:10:09 dist-test-slave-3kk6 krb5kdc[26814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:09 dist-test-slave-3kk6 krb5kdc[26814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 15:10:09.915778 26958 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:09.908684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55954 (local address 127.15.226.126:41059)
0807 15:10:09.908828 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:09.908832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:09.908844 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:10:09.908892 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:09.908895 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:09.908939 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:09.909007 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:09.909847 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.910406 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.911458 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.911675 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.911775 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:09.915648 (+ 3873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55954: BlockingRecv error: recv got EOF from 127.0.0.1:55954 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 07 15:10:09 dist-test-slave-3kk6 krb5kdc[26814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115409, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM
I20260807 15:10:09.927307 26958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:09.917764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55966 (local address 127.15.226.126:41059)
0807 15:10:09.917922 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:09.917927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:09.918061 (+ 134us) server_negotiation.cc:407] Connection header received
0807 15:10:09.918246 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:09.918249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:09.918293 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:09.918386 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:09.919168 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.919711 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.920545 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.920901 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.921022 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:09.923632 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:09.923636 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:09.923638 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:09.923670 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:09.925390 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:09.925911 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:09.925915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:09.925917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:09.926014 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:09.926336 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:09.926340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:09.926341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:09.926554 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:09.926725 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:09.926997 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 15:10:09.927127 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 15:10:09.931075 26884 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55966:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:10:09.934695 26884 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:10:09.952087 27616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:09.945198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:38203 (local address 127.0.0.1:46518)
0807 15:10:09.946172 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:09.946182 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:09.946279 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:09.946819 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:09.946824 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:09.946846 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:09.946995 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:09.947004 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.948427 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.948430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:09.949584 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:09.949598 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.951350 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.951357 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:09.951525 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:09.951583 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 15:10:09.951737 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 15:10:09.951911 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":920,"mutex_wait_us":2,"thread_start_us":109,"threads_started":1}
I20260807 15:10:09.952211 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:09.944714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47640 (local address 127.15.226.67:37875)
0807 15:10:09.945209 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:09.945226 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:09.945891 (+ 665us) server_negotiation.cc:407] Connection header received
0807 15:10:09.946085 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:09.946090 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:09.946320 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:09.946502 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:09.946908 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.948194 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.950436 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.951282 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.951395 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:09.951561 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:09.951926 (+ 365us) server_negotiation.cc:299] Negotiation successful
0807 15:10:09.952036 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":121,"threads_started":1}
I20260807 15:10:09.952441 27623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:09.945155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52032 (local address 127.15.226.66:42627)
0807 15:10:09.945631 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:09.945644 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:09.945953 (+ 309us) server_negotiation.cc:407] Connection header received
0807 15:10:09.946761 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:09.946770 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:09.946945 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:09.947123 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:09.947325 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.948395 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.950399 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.951323 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.951440 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:09.951625 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:09.952164 (+ 539us) server_negotiation.cc:299] Negotiation successful
0807 15:10:09.952295 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":138,"threads_started":1}
I20260807 15:10:09.952634 27619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:09.944379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:37875 (local address 127.0.0.1:47640)
0807 15:10:09.945700 (+ 1321us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:09.945766 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:09.945932 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:09.946499 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:09.946504 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:09.946525 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:09.946730 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:09.946738 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.948368 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.948375 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:09.950248 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:09.950261 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.951629 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.951632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:09.951763 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:09.951784 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 15:10:09.952417 (+ 633us) client_negotiation.cc:240] Negotiation successful
0807 15:10:09.952506 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1118,"mutex_wait_us":268,"spinlock_wait_cycles":2560,"thread_start_us":119,"threads_started":1}
I20260807 15:10:09.952801 27625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:09.945532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46518 (local address 127.15.226.65:38203)
0807 15:10:09.945925 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:09.945933 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:09.946306 (+ 373us) server_negotiation.cc:407] Connection header received
0807 15:10:09.946398 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:09.946406 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:09.946585 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:09.946827 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:09.947146 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.948296 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.949787 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.950824 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.950954 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:09.951138 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:09.952508 (+ 1370us) server_negotiation.cc:299] Negotiation successful
0807 15:10:09.952628 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":80,"threads_started":1}
I20260807 15:10:09.952870 27624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:09.944402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:42627 (local address 127.0.0.1:52032)
0807 15:10:09.945700 (+ 1298us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:09.945766 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:09.945932 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:09.947104 (+ 1172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:09.947107 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:09.947127 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:09.947221 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:09.947228 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.948537 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.948541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:09.950248 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:09.950261 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:09.951722 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:09.951725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:09.951825 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:09.951855 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 15:10:09.951938 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 15:10:09.952030 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1138,"mutex_wait_us":267,"spinlock_wait_cycles":11392,"thread_start_us":96,"threads_started":1}
I20260807 15:10:09.955976 27496 tablet_service.cc:1467] Processing CreateTablet for tablet 75bca950cc9d4ae99d9ff83e5e40fc15 (DEFAULT_TABLE table=test-table [id=39ce951a1e4e47dba4a9c9d36056e5df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:09.955964 27093 tablet_service.cc:1467] Processing CreateTablet for tablet 75bca950cc9d4ae99d9ff83e5e40fc15 (DEFAULT_TABLE table=test-table [id=39ce951a1e4e47dba4a9c9d36056e5df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:09.956743 27301 tablet_service.cc:1467] Processing CreateTablet for tablet 75bca950cc9d4ae99d9ff83e5e40fc15 (DEFAULT_TABLE table=test-table [id=39ce951a1e4e47dba4a9c9d36056e5df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:09.957181 27496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75bca950cc9d4ae99d9ff83e5e40fc15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:09.957191 27093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75bca950cc9d4ae99d9ff83e5e40fc15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:09.957753 27301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75bca950cc9d4ae99d9ff83e5e40fc15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:09.964623 27626 tablet_bootstrap.cc:492] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 17aafb9028354f5ea9c4302d7d3c3dbb: Bootstrap starting.
I20260807 15:10:09.965873 27628 tablet_bootstrap.cc:492] T 75bca950cc9d4ae99d9ff83e5e40fc15 P b61995e07c35456694b60e70309f2eef: Bootstrap starting.
I20260807 15:10:09.966199 27627 tablet_bootstrap.cc:492] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c: Bootstrap starting.
I20260807 15:10:09.967569 27626 tablet_bootstrap.cc:654] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 17aafb9028354f5ea9c4302d7d3c3dbb: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:09.968241 27628 tablet_bootstrap.cc:654] T 75bca950cc9d4ae99d9ff83e5e40fc15 P b61995e07c35456694b60e70309f2eef: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:09.968240 27627 tablet_bootstrap.cc:654] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:09.968434 27626 log.cc:824] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 17aafb9028354f5ea9c4302d7d3c3dbb: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:09.969110 27628 log.cc:824] T 75bca950cc9d4ae99d9ff83e5e40fc15 P b61995e07c35456694b60e70309f2eef: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:09.969110 27627 log.cc:824] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:09.970924 27626 tablet_bootstrap.cc:492] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 17aafb9028354f5ea9c4302d7d3c3dbb: No bootstrap required, opened a new log
I20260807 15:10:09.970970 27628 tablet_bootstrap.cc:492] T 75bca950cc9d4ae99d9ff83e5e40fc15 P b61995e07c35456694b60e70309f2eef: No bootstrap required, opened a new log
I20260807 15:10:09.971115 27627 tablet_bootstrap.cc:492] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c: No bootstrap required, opened a new log
I20260807 15:10:09.971158 27626 ts_tablet_manager.cc:1397] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 17aafb9028354f5ea9c4302d7d3c3dbb: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 15:10:09.971204 27628 ts_tablet_manager.cc:1397] T 75bca950cc9d4ae99d9ff83e5e40fc15 P b61995e07c35456694b60e70309f2eef: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 15:10:09.971289 27627 ts_tablet_manager.cc:1397] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 15:10:09.975104 27628 raft_consensus.cc:348] T 75bca950cc9d4ae99d9ff83e5e40fc15 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } }
I20260807 15:10:09.975507 27628 raft_consensus.cc:374] T 75bca950cc9d4ae99d9ff83e5e40fc15 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:09.975602 27628 raft_consensus.cc:729] T 75bca950cc9d4ae99d9ff83e5e40fc15 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b61995e07c35456694b60e70309f2eef, State: Initialized, Role: FOLLOWER
I20260807 15:10:09.975507 27626 raft_consensus.cc:348] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } }
I20260807 15:10:09.975854 27626 raft_consensus.cc:374] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:09.976022 27626 raft_consensus.cc:729] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17aafb9028354f5ea9c4302d7d3c3dbb, State: Initialized, Role: FOLLOWER
I20260807 15:10:09.975996 27627 raft_consensus.cc:348] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } }
I20260807 15:10:09.976277 27627 raft_consensus.cc:374] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:09.976176 27628 consensus_queue.cc:260] T 75bca950cc9d4ae99d9ff83e5e40fc15 P b61995e07c35456694b60e70309f2eef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } }
I20260807 15:10:09.976397 27627 raft_consensus.cc:729] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc5606586144eb182d4d1459833528c, State: Initialized, Role: FOLLOWER
I20260807 15:10:09.976511 27626 consensus_queue.cc:260] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 17aafb9028354f5ea9c4302d7d3c3dbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } }
I20260807 15:10:09.977082 27384 heartbeater.cc:503] Master 127.15.226.126:41059 was elected leader, sending a full tablet report...
I20260807 15:10:09.977497 27628 ts_tablet_manager.cc:1428] T 75bca950cc9d4ae99d9ff83e5e40fc15 P b61995e07c35456694b60e70309f2eef: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 15:10:09.977630 27626 ts_tablet_manager.cc:1428] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 17aafb9028354f5ea9c4302d7d3c3dbb: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 15:10:09.978010 27581 heartbeater.cc:503] Master 127.15.226.126:41059 was elected leader, sending a full tablet report...
I20260807 15:10:09.976855 27627 consensus_queue.cc:260] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } }
I20260807 15:10:09.978830 27185 heartbeater.cc:503] Master 127.15.226.126:41059 was elected leader, sending a full tablet report...
I20260807 15:10:09.979202 27627 ts_tablet_manager.cc:1428] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
W20260807 15:10:10.123729 27582 tablet.cc:2369] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 17aafb9028354f5ea9c4302d7d3c3dbb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:10:10.135367 27387 tablet.cc:2369] T 75bca950cc9d4ae99d9ff83e5e40fc15 P b61995e07c35456694b60e70309f2eef: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:10:10.144384 27186 tablet.cc:2369] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:10:10.341140 27636 raft_consensus.cc:482] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:10:10.341401 27636 raft_consensus.cc:504] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } }
I20260807 15:10:10.342846 27636 leader_election.cc:288] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17aafb9028354f5ea9c4302d7d3c3dbb (127.15.226.67:37875), b61995e07c35456694b60e70309f2eef (127.15.226.66:42627)
I20260807 15:10:10.349080 27623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:10.343284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:41711 (local address 127.15.226.66:42627)
0807 15:10:10.343465 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:10.343470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:10.343808 (+ 338us) server_negotiation.cc:407] Connection header received
0807 15:10:10.343988 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:10.343991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:10.344041 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:10.344136 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:10.344626 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.345452 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.347548 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.348199 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.348356 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:10.348419 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:10.348835 (+ 416us) server_negotiation.cc:299] Negotiation successful
0807 15:10:10.348934 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 15:10:10.349095 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:10.343479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:57707 (local address 127.15.226.67:37875)
0807 15:10:10.343658 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:10.343664 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:10.343868 (+ 204us) server_negotiation.cc:407] Connection header received
0807 15:10:10.343981 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:10.343985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:10.344041 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:10.344136 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:10.344626 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.345459 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.347548 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.348210 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.348357 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:10.348419 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:10.348851 (+ 432us) server_negotiation.cc:299] Negotiation successful
0807 15:10:10.348934 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 15:10:10.349144 27639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:10.343194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:37875 (local address 127.15.226.65:57707)
0807 15:10:10.343773 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:10.343787 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:10.343869 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:10.344233 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:10.344236 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:10.344249 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:10.344455 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:10.344463 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.345616 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.345620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:10.347361 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:10.347375 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.348384 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.348389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:10.348623 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:10.348699 (+ 76us) client_negotiation.cc:769] Sending connection context
0807 15:10:10.348794 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 15:10:10.348918 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"mutex_wait_us":329,"thread_start_us":129,"threads_started":1}
I20260807 15:10:10.349161 27640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:10.343195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:42627 (local address 127.15.226.65:41711)
0807 15:10:10.343674 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:10.343694 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:10.343862 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:10.344209 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:10.344212 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:10.344234 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:10.344455 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:10.344463 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.345616 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.345620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:10.347361 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:10.347375 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.348384 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.348389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:10.348625 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:10.348703 (+ 78us) client_negotiation.cc:769] Sending connection context
0807 15:10:10.348794 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 15:10:10.348918 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":328,"spinlock_wait_cycles":1408,"thread_start_us":148,"threads_started":1}
I20260807 15:10:10.350066 27523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75bca950cc9d4ae99d9ff83e5e40fc15" candidate_uuid: "0dc5606586144eb182d4d1459833528c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" is_pre_election: true
I20260807 15:10:10.350056 27325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75bca950cc9d4ae99d9ff83e5e40fc15" candidate_uuid: "0dc5606586144eb182d4d1459833528c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b61995e07c35456694b60e70309f2eef" is_pre_election: true
I20260807 15:10:10.350453 27325 raft_consensus.cc:2454] T 75bca950cc9d4ae99d9ff83e5e40fc15 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dc5606586144eb182d4d1459833528c in term 0.
I20260807 15:10:10.350453 27523 raft_consensus.cc:2454] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dc5606586144eb182d4d1459833528c in term 0.
I20260807 15:10:10.351179 27018 leader_election.cc:302] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0dc5606586144eb182d4d1459833528c, 17aafb9028354f5ea9c4302d7d3c3dbb, b61995e07c35456694b60e70309f2eef; no voters:
I20260807 15:10:10.351584 27636 raft_consensus.cc:2781] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:10:10.351680 27636 raft_consensus.cc:482] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:10:10.351729 27636 raft_consensus.cc:3037] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:10.353066 27636 raft_consensus.cc:504] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } }
I20260807 15:10:10.353510 27636 leader_election.cc:288] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c [CANDIDATE]: Term 1 election: Requested vote from peers 17aafb9028354f5ea9c4302d7d3c3dbb (127.15.226.67:37875), b61995e07c35456694b60e70309f2eef (127.15.226.66:42627)
I20260807 15:10:10.354125 27325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75bca950cc9d4ae99d9ff83e5e40fc15" candidate_uuid: "0dc5606586144eb182d4d1459833528c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b61995e07c35456694b60e70309f2eef"
I20260807 15:10:10.354123 27523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75bca950cc9d4ae99d9ff83e5e40fc15" candidate_uuid: "0dc5606586144eb182d4d1459833528c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb"
I20260807 15:10:10.354296 27523 raft_consensus.cc:3037] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:10.354296 27325 raft_consensus.cc:3037] T 75bca950cc9d4ae99d9ff83e5e40fc15 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:10.355834 27325 raft_consensus.cc:2454] T 75bca950cc9d4ae99d9ff83e5e40fc15 P b61995e07c35456694b60e70309f2eef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dc5606586144eb182d4d1459833528c in term 1.
I20260807 15:10:10.355834 27523 raft_consensus.cc:2454] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dc5606586144eb182d4d1459833528c in term 1.
I20260807 15:10:10.356328 27018 leader_election.cc:302] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc5606586144eb182d4d1459833528c, 17aafb9028354f5ea9c4302d7d3c3dbb; no voters:
I20260807 15:10:10.356577 27636 raft_consensus.cc:2781] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:10.356885 27636 raft_consensus.cc:686] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c [term 1 LEADER]: Becoming Leader. State: Replica: 0dc5606586144eb182d4d1459833528c, State: Running, Role: LEADER
I20260807 15:10:10.357326 27636 consensus_queue.cc:237] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } }
I20260807 15:10:10.360831 26885 catalog_manager.cc:5495] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c reported cstate change: term changed from 0 to 1, leader changed from <none> to 0dc5606586144eb182d4d1459833528c (127.15.226.65). New cstate: current_term: 1 leader_uuid: "0dc5606586144eb182d4d1459833528c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } health_report { overall_health: HEALTHY } } }
I20260807 15:10:10.406931 27625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:10.403051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46532 (local address 127.15.226.65:38203)
0807 15:10:10.403188 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:10.403197 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:10.403213 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:10:10.403349 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:10.403351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:10.403406 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:10.403503 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:10:10.403723 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.404254 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.405206 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.405445 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.405544 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:10.405959 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:10:10.406425 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:10:10.406544 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:10.406737 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 15:10:10.406812 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 15:10:10.413722 27523 raft_consensus.cc:1261] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 1 FOLLOWER]: Refusing update from remote peer 0dc5606586144eb182d4d1459833528c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:10.413715 27325 raft_consensus.cc:1261] T 75bca950cc9d4ae99d9ff83e5e40fc15 P b61995e07c35456694b60e70309f2eef [term 1 FOLLOWER]: Refusing update from remote peer 0dc5606586144eb182d4d1459833528c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:10.414584 27641 consensus_queue.cc:1035] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:10.414887 27636 consensus_queue.cc:1035] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c [LEADER]: Connected to new peer: Peer: permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:10.433728 26884 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:10.434010 26884 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:10.436887 26884 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa: Sending DeleteTablet for 3 replicas of tablet 75bca950cc9d4ae99d9ff83e5e40fc15
I20260807 15:10:10.437901 27496 tablet_service.cc:1514] Processing DeleteTablet for tablet 75bca950cc9d4ae99d9ff83e5e40fc15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:10 UTC) from {username='oryx', principal='oryx/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:47640
I20260807 15:10:10.438027 27093 tablet_service.cc:1514] Processing DeleteTablet for tablet 75bca950cc9d4ae99d9ff83e5e40fc15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:10 UTC) from {username='oryx', principal='oryx/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:46518
I20260807 15:10:10.438035 27301 tablet_service.cc:1514] Processing DeleteTablet for tablet 75bca950cc9d4ae99d9ff83e5e40fc15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:10 UTC) from {username='oryx', principal='oryx/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:52032
I20260807 15:10:10.438563 27660 tablet_replica.cc:330] T 75bca950cc9d4ae99d9ff83e5e40fc15 P b61995e07c35456694b60e70309f2eef: stopping tablet replica
I20260807 15:10:10.438875 27658 tablet_replica.cc:330] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 17aafb9028354f5ea9c4302d7d3c3dbb: stopping tablet replica
I20260807 15:10:10.439306 27658 raft_consensus.cc:2229] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:10:10.439502 27660 raft_consensus.cc:2229] T 75bca950cc9d4ae99d9ff83e5e40fc15 P b61995e07c35456694b60e70309f2eef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:10:10.439738 27658 raft_consensus.cc:2258] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:10.439674 26884 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55966:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 15:10:10.439839 27660 raft_consensus.cc:2258] T 75bca950cc9d4ae99d9ff83e5e40fc15 P b61995e07c35456694b60e70309f2eef [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 15:10:10.440187 26884 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:10:10.440378 27657 tablet_replica.cc:330] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c: stopping tablet replica
I20260807 15:10:10.440683 27657 raft_consensus.cc:2229] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:10:10.441083 27657 raft_consensus.cc:2258] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:10.441571 27660 ts_tablet_manager.cc:1905] T 75bca950cc9d4ae99d9ff83e5e40fc15 P b61995e07c35456694b60e70309f2eef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:10.444113 27658 ts_tablet_manager.cc:1905] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 17aafb9028354f5ea9c4302d7d3c3dbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:10.444849 27660 ts_tablet_manager.cc:1918] T 75bca950cc9d4ae99d9ff83e5e40fc15 P b61995e07c35456694b60e70309f2eef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:10.444975 27660 log.cc:1196] T 75bca950cc9d4ae99d9ff83e5e40fc15 P b61995e07c35456694b60e70309f2eef: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/75bca950cc9d4ae99d9ff83e5e40fc15
I20260807 15:10:10.445343 27660 ts_tablet_manager.cc:1939] T 75bca950cc9d4ae99d9ff83e5e40fc15 P b61995e07c35456694b60e70309f2eef: Deleting consensus metadata
I20260807 15:10:10.446955 26863 catalog_manager.cc:4841] TS b61995e07c35456694b60e70309f2eef (127.15.226.66:42627): tablet 75bca950cc9d4ae99d9ff83e5e40fc15 (table test-table [id=39ce951a1e4e47dba4a9c9d36056e5df]) successfully deleted
I20260807 15:10:10.447402 27301 tablet_service.cc:1467] Processing CreateTablet for tablet f1d3ca5e06bc4269b2447a654ccbf328 (DEFAULT_TABLE table=test-table [id=26500ab295c94f8898a1c0f3efbba8ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:10.447484 27093 tablet_service.cc:1467] Processing CreateTablet for tablet f1d3ca5e06bc4269b2447a654ccbf328 (DEFAULT_TABLE table=test-table [id=26500ab295c94f8898a1c0f3efbba8ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:10.447693 27301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1d3ca5e06bc4269b2447a654ccbf328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:10.447834 27093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1d3ca5e06bc4269b2447a654ccbf328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:10.447934 27658 ts_tablet_manager.cc:1918] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 17aafb9028354f5ea9c4302d7d3c3dbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:10.448093 27658 log.cc:1196] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 17aafb9028354f5ea9c4302d7d3c3dbb: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/75bca950cc9d4ae99d9ff83e5e40fc15
I20260807 15:10:10.448515 27658 ts_tablet_manager.cc:1939] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 17aafb9028354f5ea9c4302d7d3c3dbb: Deleting consensus metadata
I20260807 15:10:10.450349 27628 tablet_bootstrap.cc:492] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef: Bootstrap starting.
I20260807 15:10:10.451043 26862 catalog_manager.cc:4841] TS 17aafb9028354f5ea9c4302d7d3c3dbb (127.15.226.67:37875): tablet 75bca950cc9d4ae99d9ff83e5e40fc15 (table test-table [id=39ce951a1e4e47dba4a9c9d36056e5df]) successfully deleted
I20260807 15:10:10.451435 27628 tablet_bootstrap.cc:654] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:10.451396 27496 tablet_service.cc:1467] Processing CreateTablet for tablet f1d3ca5e06bc4269b2447a654ccbf328 (DEFAULT_TABLE table=test-table [id=26500ab295c94f8898a1c0f3efbba8ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:10.451705 27496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f1d3ca5e06bc4269b2447a654ccbf328. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:10.451972 27627 tablet_bootstrap.cc:492] T f1d3ca5e06bc4269b2447a654ccbf328 P 0dc5606586144eb182d4d1459833528c: Bootstrap starting.
I20260807 15:10:10.452998 27627 tablet_bootstrap.cc:654] T f1d3ca5e06bc4269b2447a654ccbf328 P 0dc5606586144eb182d4d1459833528c: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:10.453765 27628 tablet_bootstrap.cc:492] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef: No bootstrap required, opened a new log
I20260807 15:10:10.453852 27628 ts_tablet_manager.cc:1397] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 15:10:10.454385 27626 tablet_bootstrap.cc:492] T f1d3ca5e06bc4269b2447a654ccbf328 P 17aafb9028354f5ea9c4302d7d3c3dbb: Bootstrap starting.
I20260807 15:10:10.454578 27657 ts_tablet_manager.cc:1905] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:10.455170 27628 raft_consensus.cc:348] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:10.455361 27626 tablet_bootstrap.cc:654] T f1d3ca5e06bc4269b2447a654ccbf328 P 17aafb9028354f5ea9c4302d7d3c3dbb: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:10.455367 27628 raft_consensus.cc:374] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:10.455404 27628 raft_consensus.cc:729] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b61995e07c35456694b60e70309f2eef, State: Initialized, Role: FOLLOWER
I20260807 15:10:10.455554 27628 consensus_queue.cc:260] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:10.455885 27628 ts_tablet_manager.cc:1428] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:10:10.457056 27626 tablet_bootstrap.cc:492] T f1d3ca5e06bc4269b2447a654ccbf328 P 17aafb9028354f5ea9c4302d7d3c3dbb: No bootstrap required, opened a new log
I20260807 15:10:10.457154 27626 ts_tablet_manager.cc:1397] T f1d3ca5e06bc4269b2447a654ccbf328 P 17aafb9028354f5ea9c4302d7d3c3dbb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:10:10.457702 27657 ts_tablet_manager.cc:1918] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:10.457727 27626 raft_consensus.cc:348] T f1d3ca5e06bc4269b2447a654ccbf328 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:10.457837 27657 log.cc:1196] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-0/wal/wals/75bca950cc9d4ae99d9ff83e5e40fc15
I20260807 15:10:10.457847 27626 raft_consensus.cc:374] T f1d3ca5e06bc4269b2447a654ccbf328 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:10.457885 27626 raft_consensus.cc:729] T f1d3ca5e06bc4269b2447a654ccbf328 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17aafb9028354f5ea9c4302d7d3c3dbb, State: Initialized, Role: FOLLOWER
I20260807 15:10:10.458230 27657 ts_tablet_manager.cc:1939] T 75bca950cc9d4ae99d9ff83e5e40fc15 P 0dc5606586144eb182d4d1459833528c: Deleting consensus metadata
I20260807 15:10:10.458323 27626 consensus_queue.cc:260] T f1d3ca5e06bc4269b2447a654ccbf328 P 17aafb9028354f5ea9c4302d7d3c3dbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:10.458635 27626 ts_tablet_manager.cc:1428] T f1d3ca5e06bc4269b2447a654ccbf328 P 17aafb9028354f5ea9c4302d7d3c3dbb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:10:10.459650 26863 catalog_manager.cc:4841] TS 0dc5606586144eb182d4d1459833528c (127.15.226.65:38203): tablet 75bca950cc9d4ae99d9ff83e5e40fc15 (table test-table [id=39ce951a1e4e47dba4a9c9d36056e5df]) successfully deleted
I20260807 15:10:10.467698 27627 tablet_bootstrap.cc:492] T f1d3ca5e06bc4269b2447a654ccbf328 P 0dc5606586144eb182d4d1459833528c: No bootstrap required, opened a new log
I20260807 15:10:10.467837 27627 ts_tablet_manager.cc:1397] T f1d3ca5e06bc4269b2447a654ccbf328 P 0dc5606586144eb182d4d1459833528c: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20260807 15:10:10.468393 27627 raft_consensus.cc:348] T f1d3ca5e06bc4269b2447a654ccbf328 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:10.468544 27627 raft_consensus.cc:374] T f1d3ca5e06bc4269b2447a654ccbf328 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:10.468581 27627 raft_consensus.cc:729] T f1d3ca5e06bc4269b2447a654ccbf328 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc5606586144eb182d4d1459833528c, State: Initialized, Role: FOLLOWER
I20260807 15:10:10.468729 27627 consensus_queue.cc:260] T f1d3ca5e06bc4269b2447a654ccbf328 P 0dc5606586144eb182d4d1459833528c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:10.469008 27627 ts_tablet_manager.cc:1428] T f1d3ca5e06bc4269b2447a654ccbf328 P 0dc5606586144eb182d4d1459833528c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 15:10:10.560758 27634 raft_consensus.cc:482] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:10:10.560964 27634 raft_consensus.cc:504] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:10.562551 27634 leader_election.cc:288] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0dc5606586144eb182d4d1459833528c (127.15.226.65:38203), 17aafb9028354f5ea9c4302d7d3c3dbb (127.15.226.67:37875)
I20260807 15:10:10.568244 27625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:10.563111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:42657 (local address 127.15.226.65:38203)
0807 15:10:10.563394 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:10.563400 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:10.563678 (+ 278us) server_negotiation.cc:407] Connection header received
0807 15:10:10.563837 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:10.563840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:10.563897 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:10.563988 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:10.564388 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.564915 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.566883 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.567577 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.567744 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:10.567818 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:10.568028 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 15:10:10.568110 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202}
I20260807 15:10:10.568291 27666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:10.562886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:38203 (local address 127.15.226.66:42657)
0807 15:10:10.563563 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:10.563574 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:10.563706 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:10.564010 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:10.564012 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:10.564024 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:10.564210 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:10.564217 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.565064 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.565069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:10.566694 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:10.566710 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.567751 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.567754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:10.567863 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:10.567889 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 15:10:10.568003 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 15:10:10.568091 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"mutex_wait_us":399,"thread_start_us":104,"threads_started":1}
I20260807 15:10:10.569198 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:10.563051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:44623 (local address 127.15.226.67:37875)
0807 15:10:10.563280 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:10.563286 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:10.563435 (+ 149us) server_negotiation.cc:407] Connection header received
0807 15:10:10.563697 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:10.563702 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:10.563762 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:10.563851 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:10.564363 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.565117 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.567906 (+ 2789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.568543 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.568691 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:10.568748 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:10.568993 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 15:10:10.569095 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 15:10:10.569164 27124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1d3ca5e06bc4269b2447a654ccbf328" candidate_uuid: "b61995e07c35456694b60e70309f2eef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc5606586144eb182d4d1459833528c" is_pre_election: true
I20260807 15:10:10.569484 27124 raft_consensus.cc:2454] T f1d3ca5e06bc4269b2447a654ccbf328 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b61995e07c35456694b60e70309f2eef in term 0.
I20260807 15:10:10.569633 27665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:10.562886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:37875 (local address 127.15.226.66:44623)
0807 15:10:10.563312 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:10.563331 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:10.563537 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:10.563986 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:10.563989 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:10.564009 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:10.564210 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:10.564217 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.565247 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.565250 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:10.566696 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:10.566710 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.568686 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.568690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:10.568804 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:10.568857 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 15:10:10.569390 (+ 533us) client_negotiation.cc:240] Negotiation successful
0807 15:10:10.569484 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":161,"spinlock_wait_cycles":7552,"thread_start_us":105,"threads_started":1}
I20260807 15:10:10.570114 27523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1d3ca5e06bc4269b2447a654ccbf328" candidate_uuid: "b61995e07c35456694b60e70309f2eef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" is_pre_election: true
I20260807 15:10:10.570272 27523 raft_consensus.cc:2454] T f1d3ca5e06bc4269b2447a654ccbf328 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b61995e07c35456694b60e70309f2eef in term 0.
I20260807 15:10:10.570286 27220 leader_election.cc:302] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc5606586144eb182d4d1459833528c, b61995e07c35456694b60e70309f2eef; no voters:
I20260807 15:10:10.570739 27634 raft_consensus.cc:2781] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:10:10.570822 27634 raft_consensus.cc:482] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:10:10.570922 27634 raft_consensus.cc:3037] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:10.572453 27634 raft_consensus.cc:504] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:10.572927 27634 leader_election.cc:288] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef [CANDIDATE]: Term 1 election: Requested vote from peers 0dc5606586144eb182d4d1459833528c (127.15.226.65:38203), 17aafb9028354f5ea9c4302d7d3c3dbb (127.15.226.67:37875)
I20260807 15:10:10.573421 27523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1d3ca5e06bc4269b2447a654ccbf328" candidate_uuid: "b61995e07c35456694b60e70309f2eef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb"
I20260807 15:10:10.573443 27124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f1d3ca5e06bc4269b2447a654ccbf328" candidate_uuid: "b61995e07c35456694b60e70309f2eef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc5606586144eb182d4d1459833528c"
I20260807 15:10:10.573578 27523 raft_consensus.cc:3037] T f1d3ca5e06bc4269b2447a654ccbf328 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:10.573582 27124 raft_consensus.cc:3037] T f1d3ca5e06bc4269b2447a654ccbf328 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:10.574982 27523 raft_consensus.cc:2454] T f1d3ca5e06bc4269b2447a654ccbf328 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b61995e07c35456694b60e70309f2eef in term 1.
I20260807 15:10:10.575083 27124 raft_consensus.cc:2454] T f1d3ca5e06bc4269b2447a654ccbf328 P 0dc5606586144eb182d4d1459833528c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b61995e07c35456694b60e70309f2eef in term 1.
I20260807 15:10:10.575461 27219 leader_election.cc:302] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17aafb9028354f5ea9c4302d7d3c3dbb, b61995e07c35456694b60e70309f2eef; no voters:
I20260807 15:10:10.575801 27634 raft_consensus.cc:2781] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:10.576076 27634 raft_consensus.cc:686] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef [term 1 LEADER]: Becoming Leader. State: Replica: b61995e07c35456694b60e70309f2eef, State: Running, Role: LEADER
I20260807 15:10:10.576411 27634 consensus_queue.cc:237] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:10.579442 26885 catalog_manager.cc:5495] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef reported cstate change: term changed from 0 to 1, leader changed from <none> to b61995e07c35456694b60e70309f2eef (127.15.226.66). New cstate: current_term: 1 leader_uuid: "b61995e07c35456694b60e70309f2eef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } health_report { overall_health: UNKNOWN } } }
Aug 07 15:10:10 dist-test-slave-3kk6 krb5kdc[26814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115409, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.126@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM
I20260807 15:10:10.614850 27678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:10.604843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55968 (local address 127.15.226.126:41059)
0807 15:10:10.605206 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:10.605214 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:10.605238 (+ 24us) server_negotiation.cc:407] Connection header received
0807 15:10:10.605423 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:10.605426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:10.605471 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:10.605557 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:10.606447 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.607081 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.608012 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.608287 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.608436 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:10.611290 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:10.611294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:10.611296 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:10.611320 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:10.612832 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:10.613385 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:10.613390 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:10.613392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:10.613481 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:10.613785 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:10.613790 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:10.613792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:10.614002 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:10.614120 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:10.614456 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 15:10:10.614706 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":100,"server-negotiator.queue_time_us":284,"thread_start_us":79,"threads_started":1}
I20260807 15:10:10.614867 27676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:10.604707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:41059 (local address 127.0.0.1:55968)
0807 15:10:10.605161 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:10.605180 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:10.605303 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:10.605614 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:10.605617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:10.606179 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:10.606308 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:10.606317 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.607212 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.607216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:10.607845 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:10.607861 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.608442 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.608446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:10.608521 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:10.609251 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:10.609266 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:10.611082 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:10.612970 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:10.612987 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:10.612995 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:10.613269 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:10.613588 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:10.613593 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:10.613595 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:10.613681 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:10.614121 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:10.614131 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:10.614261 (+ 130us) client_negotiation.cc:769] Sending connection context
0807 15:10:10.614391 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 15:10:10.614604 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":212,"threads_started":1}
I20260807 15:10:10.619688 26885 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:55968:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 15:10:10.620460 26885 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:10:10.625820 27093 tablet_service.cc:1467] Processing CreateTablet for tablet 1fd7bc8bc0c641459124f29175491f69 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a80c7b69ebe94d8888581103920914d2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:10:10.626047 27496 tablet_service.cc:1467] Processing CreateTablet for tablet 1fd7bc8bc0c641459124f29175491f69 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a80c7b69ebe94d8888581103920914d2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:10:10.626039 27301 tablet_service.cc:1467] Processing CreateTablet for tablet 1fd7bc8bc0c641459124f29175491f69 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a80c7b69ebe94d8888581103920914d2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 15:10:10.626340 27093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fd7bc8bc0c641459124f29175491f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:10.626437 27301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fd7bc8bc0c641459124f29175491f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:10.626437 27496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fd7bc8bc0c641459124f29175491f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:10.628973 27627 tablet_bootstrap.cc:492] T 1fd7bc8bc0c641459124f29175491f69 P 0dc5606586144eb182d4d1459833528c: Bootstrap starting.
I20260807 15:10:10.629405 27626 tablet_bootstrap.cc:492] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb: Bootstrap starting.
I20260807 15:10:10.629769 27628 tablet_bootstrap.cc:492] T 1fd7bc8bc0c641459124f29175491f69 P b61995e07c35456694b60e70309f2eef: Bootstrap starting.
I20260807 15:10:10.629936 27627 tablet_bootstrap.cc:654] T 1fd7bc8bc0c641459124f29175491f69 P 0dc5606586144eb182d4d1459833528c: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:10.630362 27626 tablet_bootstrap.cc:654] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:10.630841 27628 tablet_bootstrap.cc:654] T 1fd7bc8bc0c641459124f29175491f69 P b61995e07c35456694b60e70309f2eef: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:10.631299 27627 tablet_bootstrap.cc:492] T 1fd7bc8bc0c641459124f29175491f69 P 0dc5606586144eb182d4d1459833528c: No bootstrap required, opened a new log
I20260807 15:10:10.631404 27627 ts_tablet_manager.cc:1397] T 1fd7bc8bc0c641459124f29175491f69 P 0dc5606586144eb182d4d1459833528c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:10:10.631765 27626 tablet_bootstrap.cc:492] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb: No bootstrap required, opened a new log
I20260807 15:10:10.631856 27626 ts_tablet_manager.cc:1397] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 15:10:10.631982 27627 raft_consensus.cc:348] T 1fd7bc8bc0c641459124f29175491f69 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:10.632133 27627 raft_consensus.cc:374] T 1fd7bc8bc0c641459124f29175491f69 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:10.632171 27627 raft_consensus.cc:729] T 1fd7bc8bc0c641459124f29175491f69 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc5606586144eb182d4d1459833528c, State: Initialized, Role: FOLLOWER
I20260807 15:10:10.632220 27628 tablet_bootstrap.cc:492] T 1fd7bc8bc0c641459124f29175491f69 P b61995e07c35456694b60e70309f2eef: No bootstrap required, opened a new log
I20260807 15:10:10.632300 27628 ts_tablet_manager.cc:1397] T 1fd7bc8bc0c641459124f29175491f69 P b61995e07c35456694b60e70309f2eef: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:10:10.632266 27626 raft_consensus.cc:348] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:10.632372 27626 raft_consensus.cc:374] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:10.632401 27626 raft_consensus.cc:729] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17aafb9028354f5ea9c4302d7d3c3dbb, State: Initialized, Role: FOLLOWER
I20260807 15:10:10.632517 27626 consensus_queue.cc:260] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:10.632754 27626 ts_tablet_manager.cc:1428] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 15:10:10.632714 27627 consensus_queue.cc:260] T 1fd7bc8bc0c641459124f29175491f69 P 0dc5606586144eb182d4d1459833528c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:10.632766 27628 raft_consensus.cc:348] T 1fd7bc8bc0c641459124f29175491f69 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:10.632890 27628 raft_consensus.cc:374] T 1fd7bc8bc0c641459124f29175491f69 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:10.632925 27628 raft_consensus.cc:729] T 1fd7bc8bc0c641459124f29175491f69 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b61995e07c35456694b60e70309f2eef, State: Initialized, Role: FOLLOWER
I20260807 15:10:10.633046 27627 ts_tablet_manager.cc:1428] T 1fd7bc8bc0c641459124f29175491f69 P 0dc5606586144eb182d4d1459833528c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:10:10.633044 27628 consensus_queue.cc:260] T 1fd7bc8bc0c641459124f29175491f69 P b61995e07c35456694b60e70309f2eef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:10.633068 27641 tablet_replica.cc:439] T 1fd7bc8bc0c641459124f29175491f69 P 0dc5606586144eb182d4d1459833528c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } } }
I20260807 15:10:10.633206 27641 tablet_replica.cc:442] T 1fd7bc8bc0c641459124f29175491f69 P 0dc5606586144eb182d4d1459833528c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:10:10.633317 27628 ts_tablet_manager.cc:1428] T 1fd7bc8bc0c641459124f29175491f69 P b61995e07c35456694b60e70309f2eef: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 15:10:10.633270 27634 tablet_replica.cc:439] T 1fd7bc8bc0c641459124f29175491f69 P b61995e07c35456694b60e70309f2eef: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } } }
I20260807 15:10:10.633394 27634 tablet_replica.cc:442] T 1fd7bc8bc0c641459124f29175491f69 P b61995e07c35456694b60e70309f2eef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:10:10.635807 27635 tablet_replica.cc:439] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } } }
I20260807 15:10:10.636009 27635 tablet_replica.cc:442] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:10:10.722579 27635 raft_consensus.cc:482] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:10:10.722826 27635 raft_consensus.cc:504] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:10.724143 27635 leader_election.cc:288] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0dc5606586144eb182d4d1459833528c (127.15.226.65:38203), b61995e07c35456694b60e70309f2eef (127.15.226.66:42627)
I20260807 15:10:10.728363 27683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:10.724457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:38203 (local address 127.15.226.67:50869)
0807 15:10:10.724775 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:10.724794 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:10.724935 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:10.725366 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:10.725368 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:10.725386 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:10.725495 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:10.725500 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.726271 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.726277 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:10.727168 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:10.727178 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.727875 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.727890 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:10.727991 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:10.728011 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 15:10:10.728107 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 15:10:10.728182 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"spinlock_wait_cycles":35584,"thread_start_us":137,"threads_started":1}
I20260807 15:10:10.728333 27625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:10.724544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:50869 (local address 127.15.226.65:38203)
0807 15:10:10.724699 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:10.724703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:10.724906 (+ 203us) server_negotiation.cc:407] Connection header received
0807 15:10:10.725107 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:10.725111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:10.725174 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:10.725232 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:10.725619 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.726151 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.727322 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.727742 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.727881 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:10.727939 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:10.728127 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 15:10:10.728217 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 15:10:10.728596 27623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:10.725081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:35143 (local address 127.15.226.66:42627)
0807 15:10:10.725221 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:10.725226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:10.725388 (+ 162us) server_negotiation.cc:407] Connection header received
0807 15:10:10.725572 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:10.725575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:10.725628 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:10.725714 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:10.726013 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.726506 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.727566 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.727978 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.728112 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:10.728164 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:10.728368 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 15:10:10.728444 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:10:10.728766 27684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:10.724900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:42627 (local address 127.15.226.67:35143)
0807 15:10:10.725262 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:10.725275 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:10.725443 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:10.725737 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:10.725740 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:10.725756 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:10.725891 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:10.725897 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.726625 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.726630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:10.727429 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:10.727438 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.728113 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.728117 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:10.728221 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:10.728242 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 15:10:10.728339 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 15:10:10.728546 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":115,"threads_started":1}
I20260807 15:10:10.728915 27124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd7bc8bc0c641459124f29175491f69" candidate_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc5606586144eb182d4d1459833528c" is_pre_election: true
I20260807 15:10:10.729116 27124 raft_consensus.cc:2454] T 1fd7bc8bc0c641459124f29175491f69 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17aafb9028354f5ea9c4302d7d3c3dbb in term 0.
I20260807 15:10:10.729586 27325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd7bc8bc0c641459124f29175491f69" candidate_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b61995e07c35456694b60e70309f2eef" is_pre_election: true
I20260807 15:10:10.729585 27420 leader_election.cc:302] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc5606586144eb182d4d1459833528c, 17aafb9028354f5ea9c4302d7d3c3dbb; no voters:
I20260807 15:10:10.729746 27325 raft_consensus.cc:2454] T 1fd7bc8bc0c641459124f29175491f69 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17aafb9028354f5ea9c4302d7d3c3dbb in term 0.
I20260807 15:10:10.729890 27635 raft_consensus.cc:2781] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:10:10.729962 27635 raft_consensus.cc:482] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:10:10.730012 27635 raft_consensus.cc:3037] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:10.731343 27635 raft_consensus.cc:504] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:10.731781 27635 leader_election.cc:288] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb [CANDIDATE]: Term 1 election: Requested vote from peers 0dc5606586144eb182d4d1459833528c (127.15.226.65:38203), b61995e07c35456694b60e70309f2eef (127.15.226.66:42627)
I20260807 15:10:10.732159 27325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd7bc8bc0c641459124f29175491f69" candidate_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b61995e07c35456694b60e70309f2eef"
I20260807 15:10:10.732179 27124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fd7bc8bc0c641459124f29175491f69" candidate_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc5606586144eb182d4d1459833528c"
I20260807 15:10:10.732300 27325 raft_consensus.cc:3037] T 1fd7bc8bc0c641459124f29175491f69 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:10.732385 27124 raft_consensus.cc:3037] T 1fd7bc8bc0c641459124f29175491f69 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:10.733469 27124 raft_consensus.cc:2454] T 1fd7bc8bc0c641459124f29175491f69 P 0dc5606586144eb182d4d1459833528c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17aafb9028354f5ea9c4302d7d3c3dbb in term 1.
I20260807 15:10:10.733454 27325 raft_consensus.cc:2454] T 1fd7bc8bc0c641459124f29175491f69 P b61995e07c35456694b60e70309f2eef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17aafb9028354f5ea9c4302d7d3c3dbb in term 1.
I20260807 15:10:10.733842 27420 leader_election.cc:302] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc5606586144eb182d4d1459833528c, 17aafb9028354f5ea9c4302d7d3c3dbb; no voters:
I20260807 15:10:10.734054 27635 raft_consensus.cc:2781] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:10.734328 27635 raft_consensus.cc:686] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 1 LEADER]: Becoming Leader. State: Replica: 17aafb9028354f5ea9c4302d7d3c3dbb, State: Running, Role: LEADER
I20260807 15:10:10.734619 27635 consensus_queue.cc:237] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:10.735998 27685 tablet_replica.cc:439] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb: TxnStatusTablet state changed. Reason: New leader 17aafb9028354f5ea9c4302d7d3c3dbb. Latest consensus state: current_term: 1 leader_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } } }
I20260807 15:10:10.736131 27685 tablet_replica.cc:442] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:10:10.736459 27687 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 15:10:10.736637 27687 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 15:10:10.737450 26885 catalog_manager.cc:5495] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb reported cstate change: term changed from 0 to 1, leader changed from <none> to 17aafb9028354f5ea9c4302d7d3c3dbb (127.15.226.67). New cstate: current_term: 1 leader_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } health_report { overall_health: HEALTHY } } }
I20260807 15:10:10.766724 27676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:10.762322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:37875 (local address 127.0.0.1:47654)
0807 15:10:10.762560 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:10.762576 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:10.762735 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:10.763112 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:10.763114 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:10.763122 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:10:10.763223 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:10.763228 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.764239 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.764241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:10.764854 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:10.764862 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.765447 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.765450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:10.765519 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:10.765567 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:10:10.766400 (+ 833us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:10:10.766413 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 15:10:10.766463 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 15:10:10.766544 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260807 15:10:10.766741 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:10.762465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47654 (local address 127.15.226.67:37875)
0807 15:10:10.762640 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:10.762666 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:10.762686 (+ 20us) server_negotiation.cc:407] Connection header received
0807 15:10:10.762900 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:10.762904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:10.762978 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:10.763093 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:10:10.763337 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.764129 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.764981 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.765327 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.765441 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:10.765717 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:10:10.766292 (+ 575us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:10:10.766411 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:10.766493 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 15:10:10.766555 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 15:10:10.770006 27325 raft_consensus.cc:1261] T 1fd7bc8bc0c641459124f29175491f69 P b61995e07c35456694b60e70309f2eef [term 1 FOLLOWER]: Refusing update from remote peer 17aafb9028354f5ea9c4302d7d3c3dbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:10.770270 27124 raft_consensus.cc:1261] T 1fd7bc8bc0c641459124f29175491f69 P 0dc5606586144eb182d4d1459833528c [term 1 FOLLOWER]: Refusing update from remote peer 17aafb9028354f5ea9c4302d7d3c3dbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:10.770615 27685 consensus_queue.cc:1035] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:10.770860 27635 consensus_queue.cc:1035] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:10.772692 27641 tablet_replica.cc:439] T 1fd7bc8bc0c641459124f29175491f69 P 0dc5606586144eb182d4d1459833528c: TxnStatusTablet state changed. Reason: New leader 17aafb9028354f5ea9c4302d7d3c3dbb. Latest consensus state: current_term: 1 leader_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } } }
I20260807 15:10:10.772811 27641 tablet_replica.cc:442] T 1fd7bc8bc0c641459124f29175491f69 P 0dc5606586144eb182d4d1459833528c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:10:10.773528 27634 tablet_replica.cc:439] T 1fd7bc8bc0c641459124f29175491f69 P b61995e07c35456694b60e70309f2eef: TxnStatusTablet state changed. Reason: New leader 17aafb9028354f5ea9c4302d7d3c3dbb. Latest consensus state: current_term: 1 leader_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } } }
I20260807 15:10:10.773650 27634 tablet_replica.cc:442] T 1fd7bc8bc0c641459124f29175491f69 P b61995e07c35456694b60e70309f2eef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:10:10.775208 27685 tablet_replica.cc:439] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } } }
I20260807 15:10:10.775331 27685 tablet_replica.cc:442] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:10:10.775638 27685 tablet_replica.cc:439] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } } }
I20260807 15:10:10.775728 27685 tablet_replica.cc:442] T 1fd7bc8bc0c641459124f29175491f69 P 17aafb9028354f5ea9c4302d7d3c3dbb: This TxnStatusTablet replica's current role is: LEADER
I20260807 15:10:10.776518 27647 mvcc.cc:205] Tried to move back new op lower bound from 7315928722508120064 to 7315928722371825664. Current Snapshot: MvccSnapshot[applied={T|T < 7315928722508120064 or (T in {7315928722508120064})}]
I20260807 15:10:10.776731 27634 tablet_replica.cc:439] T 1fd7bc8bc0c641459124f29175491f69 P b61995e07c35456694b60e70309f2eef: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } } }
I20260807 15:10:10.776834 27634 tablet_replica.cc:442] T 1fd7bc8bc0c641459124f29175491f69 P b61995e07c35456694b60e70309f2eef: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:10:10.777734 27641 tablet_replica.cc:439] T 1fd7bc8bc0c641459124f29175491f69 P 0dc5606586144eb182d4d1459833528c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } } }
I20260807 15:10:10.777870 27641 tablet_replica.cc:442] T 1fd7bc8bc0c641459124f29175491f69 P 0dc5606586144eb182d4d1459833528c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 15:10:10.779414 27644 mvcc.cc:205] Tried to move back new op lower bound from 7315928722508120064 to 7315928722371825664. Current Snapshot: MvccSnapshot[applied={T|T < 7315928722508120064 or (T in {7315928722508120064})}]
I20260807 15:10:10.785519 27623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:10.781436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52040 (local address 127.15.226.66:42627)
0807 15:10:10.781574 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:10.781579 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:10.781593 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:10:10.781672 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:10.781675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:10.781740 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:10.781854 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:10:10.782153 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.783004 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.783823 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.784158 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.784275 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:10.784518 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:10:10.785107 (+ 589us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:10:10.785238 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:10.785284 (+ 46us) server_negotiation.cc:299] Negotiation successful
0807 15:10:10.785372 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:10:10.797788 27696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:10.793717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:37875 (local address 127.15.226.66:56471)
0807 15:10:10.794063 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:10.794077 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:10.794175 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:10.794434 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:10.794436 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:10.794442 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:10:10.794563 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:10.794568 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.795649 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.795652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:10.796312 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:10.796323 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.796920 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.796923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:10.797004 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:10.797058 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:10:10.797420 (+ 362us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:10:10.797430 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 15:10:10.797530 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 15:10:10.797619 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":137,"threads_started":1}
I20260807 15:10:10.797776 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:10.793848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:56471 (local address 127.15.226.67:37875)
0807 15:10:10.793984 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:10.793989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:10.794155 (+ 166us) server_negotiation.cc:407] Connection header received
0807 15:10:10.794257 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:10.794260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:10.794316 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:10.794406 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:10:10.794692 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.795512 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.796442 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.796797 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.796903 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:10.797207 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:10:10.797307 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:10:10.797387 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:10.797555 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 15:10:10.797619 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 15:10:10.805608 27523 raft_consensus.cc:1261] T f1d3ca5e06bc4269b2447a654ccbf328 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 1 FOLLOWER]: Refusing update from remote peer b61995e07c35456694b60e70309f2eef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:10.805706 27124 raft_consensus.cc:1261] T f1d3ca5e06bc4269b2447a654ccbf328 P 0dc5606586144eb182d4d1459833528c [term 1 FOLLOWER]: Refusing update from remote peer b61995e07c35456694b60e70309f2eef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:10.806169 27668 consensus_queue.cc:1035] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef [LEADER]: Connected to new peer: Peer: permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:10.806403 27634 consensus_queue.cc:1035] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:10.814200 27649 mvcc.cc:205] Tried to move back new op lower bound from 7315928722655031296 to 7315928721724141568. Current Snapshot: MvccSnapshot[applied={T|T < 7315928722655031296}]
I20260807 15:10:10.832885 27623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:10.828094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:33925 (local address 127.15.226.66:42627)
0807 15:10:10.828229 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:10.828233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:10.828445 (+ 212us) server_negotiation.cc:407] Connection header received
0807 15:10:10.828634 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:10.828637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:10.828693 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:10.828779 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:10:10.829084 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.829618 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.831687 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.831935 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.832057 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:10.832394 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:10:10.832479 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:10:10.832562 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:10.832657 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 15:10:10.832714 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 15:10:10.832892 27701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:10.827951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:42627 (local address 127.15.226.67:33925)
0807 15:10:10.828349 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:10.828366 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:10.828487 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:10.828791 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:10.828794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:10.828803 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:10:10.828951 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:10.828959 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.829736 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.829741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:10.831545 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:10.831557 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.832088 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.832093 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:10.832194 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:10.832245 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:10:10.832571 (+ 326us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:10:10.832580 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 15:10:10.832646 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 15:10:10.832731 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":148,"threads_started":1}
I20260807 15:10:10.860762 26884 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:10.860960 26884 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:10.863209 26884 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa: Sending DeleteTablet for 3 replicas of tablet f1d3ca5e06bc4269b2447a654ccbf328
I20260807 15:10:10.864034 27093 tablet_service.cc:1514] Processing DeleteTablet for tablet f1d3ca5e06bc4269b2447a654ccbf328 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:10 UTC) from {username='oryx', principal='oryx/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:46518
I20260807 15:10:10.864034 27496 tablet_service.cc:1514] Processing DeleteTablet for tablet f1d3ca5e06bc4269b2447a654ccbf328 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:10 UTC) from {username='oryx', principal='oryx/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:47640
I20260807 15:10:10.864277 27657 tablet_replica.cc:330] T f1d3ca5e06bc4269b2447a654ccbf328 P 0dc5606586144eb182d4d1459833528c: stopping tablet replica
I20260807 15:10:10.864277 27658 tablet_replica.cc:330] T f1d3ca5e06bc4269b2447a654ccbf328 P 17aafb9028354f5ea9c4302d7d3c3dbb: stopping tablet replica
I20260807 15:10:10.864435 27658 raft_consensus.cc:2229] T f1d3ca5e06bc4269b2447a654ccbf328 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:10:10.864444 27657 raft_consensus.cc:2229] T f1d3ca5e06bc4269b2447a654ccbf328 P 0dc5606586144eb182d4d1459833528c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:10:10.864447 27301 tablet_service.cc:1514] Processing DeleteTablet for tablet f1d3ca5e06bc4269b2447a654ccbf328 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:10 UTC) from {username='oryx', principal='oryx/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:52032
I20260807 15:10:10.864581 27658 raft_consensus.cc:2258] T f1d3ca5e06bc4269b2447a654ccbf328 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:10.864581 27657 raft_consensus.cc:2258] T f1d3ca5e06bc4269b2447a654ccbf328 P 0dc5606586144eb182d4d1459833528c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:10.864661 27660 tablet_replica.cc:330] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef: stopping tablet replica
I20260807 15:10:10.864782 27660 raft_consensus.cc:2229] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:10:10.865134 27660 raft_consensus.cc:2258] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:10.865468 27657 ts_tablet_manager.cc:1905] T f1d3ca5e06bc4269b2447a654ccbf328 P 0dc5606586144eb182d4d1459833528c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:10.866056 27658 ts_tablet_manager.cc:1905] T f1d3ca5e06bc4269b2447a654ccbf328 P 17aafb9028354f5ea9c4302d7d3c3dbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:10.868772 27657 ts_tablet_manager.cc:1918] T f1d3ca5e06bc4269b2447a654ccbf328 P 0dc5606586144eb182d4d1459833528c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:10:10.868857 27657 log.cc:1196] T f1d3ca5e06bc4269b2447a654ccbf328 P 0dc5606586144eb182d4d1459833528c: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-0/wal/wals/f1d3ca5e06bc4269b2447a654ccbf328
I20260807 15:10:10.869153 27657 ts_tablet_manager.cc:1939] T f1d3ca5e06bc4269b2447a654ccbf328 P 0dc5606586144eb182d4d1459833528c: Deleting consensus metadata
I20260807 15:10:10.870038 26863 catalog_manager.cc:4841] TS 0dc5606586144eb182d4d1459833528c (127.15.226.65:38203): tablet f1d3ca5e06bc4269b2447a654ccbf328 (table test-table [id=26500ab295c94f8898a1c0f3efbba8ac]) successfully deleted
I20260807 15:10:10.871654 27660 ts_tablet_manager.cc:1905] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:10.875312 27660 ts_tablet_manager.cc:1918] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:10:10.875422 27660 log.cc:1196] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/f1d3ca5e06bc4269b2447a654ccbf328
I20260807 15:10:10.875735 27660 ts_tablet_manager.cc:1939] T f1d3ca5e06bc4269b2447a654ccbf328 P b61995e07c35456694b60e70309f2eef: Deleting consensus metadata
I20260807 15:10:10.876636 26863 catalog_manager.cc:4841] TS b61995e07c35456694b60e70309f2eef (127.15.226.66:42627): tablet f1d3ca5e06bc4269b2447a654ccbf328 (table test-table [id=26500ab295c94f8898a1c0f3efbba8ac]) successfully deleted
I20260807 15:10:10.882953 27658 ts_tablet_manager.cc:1918] T f1d3ca5e06bc4269b2447a654ccbf328 P 17aafb9028354f5ea9c4302d7d3c3dbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:10:10.883190 27658 log.cc:1196] T f1d3ca5e06bc4269b2447a654ccbf328 P 17aafb9028354f5ea9c4302d7d3c3dbb: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/f1d3ca5e06bc4269b2447a654ccbf328
I20260807 15:10:10.883611 27658 ts_tablet_manager.cc:1939] T f1d3ca5e06bc4269b2447a654ccbf328 P 17aafb9028354f5ea9c4302d7d3c3dbb: Deleting consensus metadata
I20260807 15:10:10.884593 26862 catalog_manager.cc:4841] TS 17aafb9028354f5ea9c4302d7d3c3dbb (127.15.226.67:37875): tablet f1d3ca5e06bc4269b2447a654ccbf328 (table test-table [id=26500ab295c94f8898a1c0f3efbba8ac]) successfully deleted
Aug 07 15:10:10 dist-test-slave-3kk6 krb5kdc[26814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 15:10:10 dist-test-slave-3kk6 krb5kdc[26814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:10 dist-test-slave-3kk6 krb5kdc[26814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 15:10:10.911872 27678 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:10.903930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55982 (local address 127.15.226.126:41059)
0807 15:10:10.904109 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:10.904114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:10.904655 (+ 541us) server_negotiation.cc:407] Connection header received
0807 15:10:10.904856 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:10.904859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:10.904924 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:10.904983 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:10.906045 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.906873 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.907866 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.908120 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.908250 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:10.911729 (+ 3479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55982: BlockingRecv error: recv got EOF from 127.0.0.1:55982 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 07 15:10:10 dist-test-slave-3kk6 krb5kdc[26814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM
I20260807 15:10:10.923520 27678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:10.913698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55998 (local address 127.15.226.126:41059)
0807 15:10:10.913826 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:10.913833 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:10.914324 (+ 491us) server_negotiation.cc:407] Connection header received
0807 15:10:10.914485 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:10.914488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:10.914544 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:10.914634 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:10.915306 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.915843 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.916688 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:10.916950 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:10.917067 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:10.919763 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:10.919766 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:10.919769 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:10.919802 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:10.921418 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:10.921918 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:10.921921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:10.921922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:10.922037 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:10.922381 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:10.922387 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:10.922389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:10.922601 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:10.922823 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:10.923145 (+ 322us) server_negotiation.cc:299] Negotiation successful
0807 15:10:10.923304 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 15:10:10.927115 26884 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:10:10.927649 26884 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:10:10.932928 27093 tablet_service.cc:1467] Processing CreateTablet for tablet 85872193af534db4838b6c8c9e0baac4 (DEFAULT_TABLE table=test-table [id=6bd07f97c7344b4794be98db38146675]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:10.933143 27301 tablet_service.cc:1467] Processing CreateTablet for tablet 85872193af534db4838b6c8c9e0baac4 (DEFAULT_TABLE table=test-table [id=6bd07f97c7344b4794be98db38146675]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:10.933319 27093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85872193af534db4838b6c8c9e0baac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:10.933452 27301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85872193af534db4838b6c8c9e0baac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:10.935484 27496 tablet_service.cc:1467] Processing CreateTablet for tablet 85872193af534db4838b6c8c9e0baac4 (DEFAULT_TABLE table=test-table [id=6bd07f97c7344b4794be98db38146675]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:10.935815 27496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85872193af534db4838b6c8c9e0baac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:10.936110 27627 tablet_bootstrap.cc:492] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c: Bootstrap starting.
I20260807 15:10:10.937042 27627 tablet_bootstrap.cc:654] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:10.938515 27626 tablet_bootstrap.cc:492] T 85872193af534db4838b6c8c9e0baac4 P 17aafb9028354f5ea9c4302d7d3c3dbb: Bootstrap starting.
I20260807 15:10:10.939520 27626 tablet_bootstrap.cc:654] T 85872193af534db4838b6c8c9e0baac4 P 17aafb9028354f5ea9c4302d7d3c3dbb: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:10.940584 27628 tablet_bootstrap.cc:492] T 85872193af534db4838b6c8c9e0baac4 P b61995e07c35456694b60e70309f2eef: Bootstrap starting.
I20260807 15:10:10.941685 27628 tablet_bootstrap.cc:654] T 85872193af534db4838b6c8c9e0baac4 P b61995e07c35456694b60e70309f2eef: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:10.942127 27627 tablet_bootstrap.cc:492] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c: No bootstrap required, opened a new log
I20260807 15:10:10.942225 27627 ts_tablet_manager.cc:1397] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 15:10:10.942792 27627 raft_consensus.cc:348] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:10.942942 27627 raft_consensus.cc:374] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:10.943001 27627 raft_consensus.cc:729] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc5606586144eb182d4d1459833528c, State: Initialized, Role: FOLLOWER
I20260807 15:10:10.943145 27627 consensus_queue.cc:260] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:10.943445 27627 ts_tablet_manager.cc:1428] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 15:10:10.944640 27626 tablet_bootstrap.cc:492] T 85872193af534db4838b6c8c9e0baac4 P 17aafb9028354f5ea9c4302d7d3c3dbb: No bootstrap required, opened a new log
I20260807 15:10:10.944748 27626 ts_tablet_manager.cc:1397] T 85872193af534db4838b6c8c9e0baac4 P 17aafb9028354f5ea9c4302d7d3c3dbb: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 15:10:10.944821 27628 tablet_bootstrap.cc:492] T 85872193af534db4838b6c8c9e0baac4 P b61995e07c35456694b60e70309f2eef: No bootstrap required, opened a new log
I20260807 15:10:10.944921 27628 ts_tablet_manager.cc:1397] T 85872193af534db4838b6c8c9e0baac4 P b61995e07c35456694b60e70309f2eef: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 15:10:10.945211 27626 raft_consensus.cc:348] T 85872193af534db4838b6c8c9e0baac4 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:10.945345 27626 raft_consensus.cc:374] T 85872193af534db4838b6c8c9e0baac4 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:10.945402 27626 raft_consensus.cc:729] T 85872193af534db4838b6c8c9e0baac4 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17aafb9028354f5ea9c4302d7d3c3dbb, State: Initialized, Role: FOLLOWER
I20260807 15:10:10.945416 27628 raft_consensus.cc:348] T 85872193af534db4838b6c8c9e0baac4 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:10.945530 27628 raft_consensus.cc:374] T 85872193af534db4838b6c8c9e0baac4 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:10.945561 27628 raft_consensus.cc:729] T 85872193af534db4838b6c8c9e0baac4 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b61995e07c35456694b60e70309f2eef, State: Initialized, Role: FOLLOWER
I20260807 15:10:10.945542 27626 consensus_queue.cc:260] T 85872193af534db4838b6c8c9e0baac4 P 17aafb9028354f5ea9c4302d7d3c3dbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:10.945686 27628 consensus_queue.cc:260] T 85872193af534db4838b6c8c9e0baac4 P b61995e07c35456694b60e70309f2eef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:10.945966 27628 ts_tablet_manager.cc:1428] T 85872193af534db4838b6c8c9e0baac4 P b61995e07c35456694b60e70309f2eef: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 15:10:10.946110 27626 ts_tablet_manager.cc:1428] T 85872193af534db4838b6c8c9e0baac4 P 17aafb9028354f5ea9c4302d7d3c3dbb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:10:11.052170 27641 raft_consensus.cc:482] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:10:11.052363 27641 raft_consensus.cc:504] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:11.053025 27641 leader_election.cc:288] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b61995e07c35456694b60e70309f2eef (127.15.226.66:42627), 17aafb9028354f5ea9c4302d7d3c3dbb (127.15.226.67:37875)
I20260807 15:10:11.053594 27523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85872193af534db4838b6c8c9e0baac4" candidate_uuid: "0dc5606586144eb182d4d1459833528c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" is_pre_election: true
I20260807 15:10:11.053594 27325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85872193af534db4838b6c8c9e0baac4" candidate_uuid: "0dc5606586144eb182d4d1459833528c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b61995e07c35456694b60e70309f2eef" is_pre_election: true
I20260807 15:10:11.053776 27325 raft_consensus.cc:2454] T 85872193af534db4838b6c8c9e0baac4 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dc5606586144eb182d4d1459833528c in term 0.
I20260807 15:10:11.053776 27523 raft_consensus.cc:2454] T 85872193af534db4838b6c8c9e0baac4 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dc5606586144eb182d4d1459833528c in term 0.
I20260807 15:10:11.054129 27019 leader_election.cc:302] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc5606586144eb182d4d1459833528c, b61995e07c35456694b60e70309f2eef; no voters:
I20260807 15:10:11.054373 27641 raft_consensus.cc:2781] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:10:11.054466 27641 raft_consensus.cc:482] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:10:11.054522 27641 raft_consensus.cc:3037] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:11.055739 27641 raft_consensus.cc:504] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:11.056154 27641 leader_election.cc:288] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c [CANDIDATE]: Term 1 election: Requested vote from peers b61995e07c35456694b60e70309f2eef (127.15.226.66:42627), 17aafb9028354f5ea9c4302d7d3c3dbb (127.15.226.67:37875)
I20260807 15:10:11.056565 27523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85872193af534db4838b6c8c9e0baac4" candidate_uuid: "0dc5606586144eb182d4d1459833528c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb"
I20260807 15:10:11.056567 27325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85872193af534db4838b6c8c9e0baac4" candidate_uuid: "0dc5606586144eb182d4d1459833528c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b61995e07c35456694b60e70309f2eef"
I20260807 15:10:11.056708 27325 raft_consensus.cc:3037] T 85872193af534db4838b6c8c9e0baac4 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:11.056720 27523 raft_consensus.cc:3037] T 85872193af534db4838b6c8c9e0baac4 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:11.057734 27523 raft_consensus.cc:2454] T 85872193af534db4838b6c8c9e0baac4 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dc5606586144eb182d4d1459833528c in term 1.
I20260807 15:10:11.057902 27325 raft_consensus.cc:2454] T 85872193af534db4838b6c8c9e0baac4 P b61995e07c35456694b60e70309f2eef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dc5606586144eb182d4d1459833528c in term 1.
I20260807 15:10:11.058115 27018 leader_election.cc:302] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc5606586144eb182d4d1459833528c, 17aafb9028354f5ea9c4302d7d3c3dbb; no voters:
I20260807 15:10:11.058315 27641 raft_consensus.cc:2781] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:11.058542 27641 raft_consensus.cc:686] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c [term 1 LEADER]: Becoming Leader. State: Replica: 0dc5606586144eb182d4d1459833528c, State: Running, Role: LEADER
I20260807 15:10:11.058730 27641 consensus_queue.cc:237] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:11.060616 26884 catalog_manager.cc:5495] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c reported cstate change: term changed from 0 to 1, leader changed from <none> to 0dc5606586144eb182d4d1459833528c (127.15.226.65). New cstate: current_term: 1 leader_uuid: "0dc5606586144eb182d4d1459833528c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } health_report { overall_health: UNKNOWN } } }
I20260807 15:10:11.078183 27625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:11.074848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46548 (local address 127.15.226.65:38203)
0807 15:10:11.075026 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:11.075031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:11.075052 (+ 21us) server_negotiation.cc:407] Connection header received
0807 15:10:11.075123 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:11.075126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:11.075202 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:11.075292 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:10:11.075574 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:11.076117 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:11.077012 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:11.077249 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:11.077364 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:11.077659 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:10:11.077739 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:10:11.077826 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:11.077953 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 15:10:11.078015 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 15:10:11.080677 27325 raft_consensus.cc:1261] T 85872193af534db4838b6c8c9e0baac4 P b61995e07c35456694b60e70309f2eef [term 1 FOLLOWER]: Refusing update from remote peer 0dc5606586144eb182d4d1459833528c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:11.080744 27523 raft_consensus.cc:1261] T 85872193af534db4838b6c8c9e0baac4 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 1 FOLLOWER]: Refusing update from remote peer 0dc5606586144eb182d4d1459833528c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:11.081202 27641 consensus_queue.cc:1035] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:11.081372 27728 consensus_queue.cc:1035] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c [LEADER]: Connected to new peer: Peer: permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:11.090802 26884 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:11.090956 26884 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:11.093073 26884 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa: Sending DeleteTablet for 3 replicas of tablet 85872193af534db4838b6c8c9e0baac4
I20260807 15:10:11.093842 27301 tablet_service.cc:1514] Processing DeleteTablet for tablet 85872193af534db4838b6c8c9e0baac4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:11 UTC) from {username='oryx', principal='oryx/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:52032
I20260807 15:10:11.093817 27093 tablet_service.cc:1514] Processing DeleteTablet for tablet 85872193af534db4838b6c8c9e0baac4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:11 UTC) from {username='oryx', principal='oryx/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:46518
I20260807 15:10:11.093883 27496 tablet_service.cc:1514] Processing DeleteTablet for tablet 85872193af534db4838b6c8c9e0baac4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:11 UTC) from {username='oryx', principal='oryx/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:47640
I20260807 15:10:11.094091 27658 tablet_replica.cc:330] T 85872193af534db4838b6c8c9e0baac4 P 17aafb9028354f5ea9c4302d7d3c3dbb: stopping tablet replica
I20260807 15:10:11.094090 27660 tablet_replica.cc:330] T 85872193af534db4838b6c8c9e0baac4 P b61995e07c35456694b60e70309f2eef: stopping tablet replica
I20260807 15:10:11.094228 27660 raft_consensus.cc:2229] T 85872193af534db4838b6c8c9e0baac4 P b61995e07c35456694b60e70309f2eef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:10:11.094228 27658 raft_consensus.cc:2229] T 85872193af534db4838b6c8c9e0baac4 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:10:11.094383 27660 raft_consensus.cc:2258] T 85872193af534db4838b6c8c9e0baac4 P b61995e07c35456694b60e70309f2eef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:11.094381 27658 raft_consensus.cc:2258] T 85872193af534db4838b6c8c9e0baac4 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:11.094583 27657 tablet_replica.cc:330] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c: stopping tablet replica
I20260807 15:10:11.094780 27657 raft_consensus.cc:2229] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:10:11.095039 27657 raft_consensus.cc:2258] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:11.095055 26884 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 15:10:11.095242 27660 ts_tablet_manager.cc:1905] T 85872193af534db4838b6c8c9e0baac4 P b61995e07c35456694b60e70309f2eef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:11.095481 27658 ts_tablet_manager.cc:1905] T 85872193af534db4838b6c8c9e0baac4 P 17aafb9028354f5ea9c4302d7d3c3dbb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 15:10:11.095520 26884 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:10:11.098086 27660 ts_tablet_manager.cc:1918] T 85872193af534db4838b6c8c9e0baac4 P b61995e07c35456694b60e70309f2eef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:11.098134 27658 ts_tablet_manager.cc:1918] T 85872193af534db4838b6c8c9e0baac4 P 17aafb9028354f5ea9c4302d7d3c3dbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:11.098173 27660 log.cc:1196] T 85872193af534db4838b6c8c9e0baac4 P b61995e07c35456694b60e70309f2eef: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/85872193af534db4838b6c8c9e0baac4
I20260807 15:10:11.098228 27658 log.cc:1196] T 85872193af534db4838b6c8c9e0baac4 P 17aafb9028354f5ea9c4302d7d3c3dbb: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/85872193af534db4838b6c8c9e0baac4
I20260807 15:10:11.098489 27660 ts_tablet_manager.cc:1939] T 85872193af534db4838b6c8c9e0baac4 P b61995e07c35456694b60e70309f2eef: Deleting consensus metadata
I20260807 15:10:11.098531 27658 ts_tablet_manager.cc:1939] T 85872193af534db4838b6c8c9e0baac4 P 17aafb9028354f5ea9c4302d7d3c3dbb: Deleting consensus metadata
I20260807 15:10:11.099529 27657 ts_tablet_manager.cc:1905] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:11.099757 26863 catalog_manager.cc:4841] TS b61995e07c35456694b60e70309f2eef (127.15.226.66:42627): tablet 85872193af534db4838b6c8c9e0baac4 (table test-table [id=6bd07f97c7344b4794be98db38146675]) successfully deleted
I20260807 15:10:11.101312 27093 tablet_service.cc:1467] Processing CreateTablet for tablet 0889384793424e09ae21d8d5d84af957 (DEFAULT_TABLE table=test-table [id=f73cffd73974498488fcb5dce858d90c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:11.101325 27301 tablet_service.cc:1467] Processing CreateTablet for tablet 0889384793424e09ae21d8d5d84af957 (DEFAULT_TABLE table=test-table [id=f73cffd73974498488fcb5dce858d90c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:11.101601 27093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0889384793424e09ae21d8d5d84af957. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:11.101601 27301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0889384793424e09ae21d8d5d84af957. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:11.102111 27657 ts_tablet_manager.cc:1918] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:11.102224 27657 log.cc:1196] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-0/wal/wals/85872193af534db4838b6c8c9e0baac4
I20260807 15:10:11.102520 27657 ts_tablet_manager.cc:1939] T 85872193af534db4838b6c8c9e0baac4 P 0dc5606586144eb182d4d1459833528c: Deleting consensus metadata
I20260807 15:10:11.103261 27496 tablet_service.cc:1467] Processing CreateTablet for tablet 0889384793424e09ae21d8d5d84af957 (DEFAULT_TABLE table=test-table [id=f73cffd73974498488fcb5dce858d90c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:11.103392 26863 catalog_manager.cc:4841] TS 0dc5606586144eb182d4d1459833528c (127.15.226.65:38203): tablet 85872193af534db4838b6c8c9e0baac4 (table test-table [id=6bd07f97c7344b4794be98db38146675]) successfully deleted
I20260807 15:10:11.103600 27496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0889384793424e09ae21d8d5d84af957. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:11.104081 27627 tablet_bootstrap.cc:492] T 0889384793424e09ae21d8d5d84af957 P 0dc5606586144eb182d4d1459833528c: Bootstrap starting.
I20260807 15:10:11.105067 27627 tablet_bootstrap.cc:654] T 0889384793424e09ae21d8d5d84af957 P 0dc5606586144eb182d4d1459833528c: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:11.106069 27626 tablet_bootstrap.cc:492] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb: Bootstrap starting.
I20260807 15:10:11.106269 27627 tablet_bootstrap.cc:492] T 0889384793424e09ae21d8d5d84af957 P 0dc5606586144eb182d4d1459833528c: No bootstrap required, opened a new log
I20260807 15:10:11.106380 27627 ts_tablet_manager.cc:1397] T 0889384793424e09ae21d8d5d84af957 P 0dc5606586144eb182d4d1459833528c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 15:10:11.106420 26862 catalog_manager.cc:4841] TS 17aafb9028354f5ea9c4302d7d3c3dbb (127.15.226.67:37875): tablet 85872193af534db4838b6c8c9e0baac4 (table test-table [id=6bd07f97c7344b4794be98db38146675]) successfully deleted
I20260807 15:10:11.106817 27627 raft_consensus.cc:348] T 0889384793424e09ae21d8d5d84af957 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:11.106959 27627 raft_consensus.cc:374] T 0889384793424e09ae21d8d5d84af957 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:11.106983 27626 tablet_bootstrap.cc:654] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:11.107003 27627 raft_consensus.cc:729] T 0889384793424e09ae21d8d5d84af957 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dc5606586144eb182d4d1459833528c, State: Initialized, Role: FOLLOWER
I20260807 15:10:11.107141 27627 consensus_queue.cc:260] T 0889384793424e09ae21d8d5d84af957 P 0dc5606586144eb182d4d1459833528c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:11.107511 27627 ts_tablet_manager.cc:1428] T 0889384793424e09ae21d8d5d84af957 P 0dc5606586144eb182d4d1459833528c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 15:10:11.108428 27628 tablet_bootstrap.cc:492] T 0889384793424e09ae21d8d5d84af957 P b61995e07c35456694b60e70309f2eef: Bootstrap starting.
I20260807 15:10:11.109338 27628 tablet_bootstrap.cc:654] T 0889384793424e09ae21d8d5d84af957 P b61995e07c35456694b60e70309f2eef: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:11.109411 27626 tablet_bootstrap.cc:492] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb: No bootstrap required, opened a new log
I20260807 15:10:11.109494 27626 ts_tablet_manager.cc:1397] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 15:10:11.109990 27626 raft_consensus.cc:348] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:11.110136 27626 raft_consensus.cc:374] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:11.110186 27626 raft_consensus.cc:729] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17aafb9028354f5ea9c4302d7d3c3dbb, State: Initialized, Role: FOLLOWER
I20260807 15:10:11.110334 27626 consensus_queue.cc:260] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:11.110598 27628 tablet_bootstrap.cc:492] T 0889384793424e09ae21d8d5d84af957 P b61995e07c35456694b60e70309f2eef: No bootstrap required, opened a new log
I20260807 15:10:11.110633 27626 ts_tablet_manager.cc:1428] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 15:10:11.110725 27628 ts_tablet_manager.cc:1397] T 0889384793424e09ae21d8d5d84af957 P b61995e07c35456694b60e70309f2eef: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 15:10:11.111244 27628 raft_consensus.cc:348] T 0889384793424e09ae21d8d5d84af957 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:11.111394 27628 raft_consensus.cc:374] T 0889384793424e09ae21d8d5d84af957 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:11.111439 27628 raft_consensus.cc:729] T 0889384793424e09ae21d8d5d84af957 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b61995e07c35456694b60e70309f2eef, State: Initialized, Role: FOLLOWER
I20260807 15:10:11.111589 27628 consensus_queue.cc:260] T 0889384793424e09ae21d8d5d84af957 P b61995e07c35456694b60e70309f2eef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:11.112000 27628 ts_tablet_manager.cc:1428] T 0889384793424e09ae21d8d5d84af957 P b61995e07c35456694b60e70309f2eef: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 15:10:11.189880 27635 raft_consensus.cc:482] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:10:11.190045 27635 raft_consensus.cc:504] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:11.190459 27635 leader_election.cc:288] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0dc5606586144eb182d4d1459833528c (127.15.226.65:38203), b61995e07c35456694b60e70309f2eef (127.15.226.66:42627)
I20260807 15:10:11.191082 27124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0889384793424e09ae21d8d5d84af957" candidate_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc5606586144eb182d4d1459833528c" is_pre_election: true
I20260807 15:10:11.191280 27124 raft_consensus.cc:2454] T 0889384793424e09ae21d8d5d84af957 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17aafb9028354f5ea9c4302d7d3c3dbb in term 0.
I20260807 15:10:11.191197 27325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0889384793424e09ae21d8d5d84af957" candidate_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b61995e07c35456694b60e70309f2eef" is_pre_election: true
I20260807 15:10:11.191435 27325 raft_consensus.cc:2454] T 0889384793424e09ae21d8d5d84af957 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17aafb9028354f5ea9c4302d7d3c3dbb in term 0.
I20260807 15:10:11.191670 27420 leader_election.cc:302] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc5606586144eb182d4d1459833528c, 17aafb9028354f5ea9c4302d7d3c3dbb; no voters:
I20260807 15:10:11.191917 27635 raft_consensus.cc:2781] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:10:11.191999 27635 raft_consensus.cc:482] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:10:11.192039 27635 raft_consensus.cc:3037] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:11.193308 27635 raft_consensus.cc:504] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:11.193684 27635 leader_election.cc:288] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb [CANDIDATE]: Term 1 election: Requested vote from peers 0dc5606586144eb182d4d1459833528c (127.15.226.65:38203), b61995e07c35456694b60e70309f2eef (127.15.226.66:42627)
I20260807 15:10:11.194211 27124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0889384793424e09ae21d8d5d84af957" candidate_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0dc5606586144eb182d4d1459833528c"
I20260807 15:10:11.194291 27325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0889384793424e09ae21d8d5d84af957" candidate_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b61995e07c35456694b60e70309f2eef"
I20260807 15:10:11.194387 27124 raft_consensus.cc:3037] T 0889384793424e09ae21d8d5d84af957 P 0dc5606586144eb182d4d1459833528c [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:11.194464 27325 raft_consensus.cc:3037] T 0889384793424e09ae21d8d5d84af957 P b61995e07c35456694b60e70309f2eef [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:11.195618 27124 raft_consensus.cc:2454] T 0889384793424e09ae21d8d5d84af957 P 0dc5606586144eb182d4d1459833528c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17aafb9028354f5ea9c4302d7d3c3dbb in term 1.
I20260807 15:10:11.195616 27325 raft_consensus.cc:2454] T 0889384793424e09ae21d8d5d84af957 P b61995e07c35456694b60e70309f2eef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17aafb9028354f5ea9c4302d7d3c3dbb in term 1.
I20260807 15:10:11.196024 27420 leader_election.cc:302] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dc5606586144eb182d4d1459833528c, 17aafb9028354f5ea9c4302d7d3c3dbb; no voters:
I20260807 15:10:11.196317 27635 raft_consensus.cc:2781] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:11.196451 27635 raft_consensus.cc:686] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 1 LEADER]: Becoming Leader. State: Replica: 17aafb9028354f5ea9c4302d7d3c3dbb, State: Running, Role: LEADER
I20260807 15:10:11.196624 27635 consensus_queue.cc:237] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } }
I20260807 15:10:11.198575 26884 catalog_manager.cc:5495] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb reported cstate change: term changed from 0 to 1, leader changed from <none> to 17aafb9028354f5ea9c4302d7d3c3dbb (127.15.226.67). New cstate: current_term: 1 leader_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17aafb9028354f5ea9c4302d7d3c3dbb" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 37875 } health_report { overall_health: HEALTHY } } }
I20260807 15:10:11.222748 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:11.219290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47668 (local address 127.15.226.67:37875)
0807 15:10:11.219424 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:11.219429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:11.219498 (+ 69us) server_negotiation.cc:407] Connection header received
0807 15:10:11.219672 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:11.219675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:11.219731 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:11.219816 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:10:11.220076 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:11.220614 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:11.221589 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:11.221826 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:11.221954 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:11.222234 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:10:11.222323 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:10:11.222409 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:11.222509 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 15:10:11.222570 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 15:10:11.233997 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:11.230744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:50705 (local address 127.15.226.67:37875)
0807 15:10:11.230870 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:11.230875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:11.230889 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:10:11.231050 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:11.231054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:11.231096 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:11.231183 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:10:11.231439 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:11.231972 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:11.232825 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:11.233066 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:11.233175 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:11.233468 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:10:11.233551 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:10:11.233636 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:11.233721 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 15:10:11.233795 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":58}
I20260807 15:10:11.233997 27701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:11.230586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:37875 (local address 127.15.226.67:50705)
0807 15:10:11.230773 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:11.230809 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:11.230946 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:11.231190 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:11.231193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:11.231199 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 15:10:11.231330 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:11.231336 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:11.232087 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:11.232092 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:11.232698 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:11.232707 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:11.233187 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:11.233190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:11.233309 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:11.233334 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 15:10:11.233639 (+ 305us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 15:10:11.233647 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 15:10:11.233715 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 15:10:11.233835 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":23}
I20260807 15:10:11.240638 27325 raft_consensus.cc:1261] T 0889384793424e09ae21d8d5d84af957 P b61995e07c35456694b60e70309f2eef [term 1 FOLLOWER]: Refusing update from remote peer 17aafb9028354f5ea9c4302d7d3c3dbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:11.240718 27124 raft_consensus.cc:1261] T 0889384793424e09ae21d8d5d84af957 P 0dc5606586144eb182d4d1459833528c [term 1 FOLLOWER]: Refusing update from remote peer 17aafb9028354f5ea9c4302d7d3c3dbb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:11.241109 27703 consensus_queue.cc:1035] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "b61995e07c35456694b60e70309f2eef" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:11.241255 27635 consensus_queue.cc:1035] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dc5606586144eb182d4d1459833528c" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:11.279585 26880 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:11.279795 26880 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:11.282286 26880 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 742bd21cffe34aef89250a542ac1e8fa: Sending DeleteTablet for 3 replicas of tablet 0889384793424e09ae21d8d5d84af957
I20260807 15:10:11.283363 27093 tablet_service.cc:1514] Processing DeleteTablet for tablet 0889384793424e09ae21d8d5d84af957 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:11 UTC) from {username='oryx', principal='oryx/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:46518
I20260807 15:10:11.283648 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 26998
I20260807 15:10:11.283644 27301 tablet_service.cc:1514] Processing DeleteTablet for tablet 0889384793424e09ae21d8d5d84af957 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:11 UTC) from {username='oryx', principal='oryx/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:52032
I20260807 15:10:11.283864 27657 tablet_replica.cc:330] T 0889384793424e09ae21d8d5d84af957 P 0dc5606586144eb182d4d1459833528c: stopping tablet replica
I20260807 15:10:11.284021 27657 raft_consensus.cc:2229] T 0889384793424e09ae21d8d5d84af957 P 0dc5606586144eb182d4d1459833528c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:10:11.284168 27657 raft_consensus.cc:2258] T 0889384793424e09ae21d8d5d84af957 P 0dc5606586144eb182d4d1459833528c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:11.289448 27660 tablet_replica.cc:330] T 0889384793424e09ae21d8d5d84af957 P b61995e07c35456694b60e70309f2eef: stopping tablet replica
I20260807 15:10:11.289685 27660 raft_consensus.cc:2229] T 0889384793424e09ae21d8d5d84af957 P b61995e07c35456694b60e70309f2eef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:10:11.289862 27660 raft_consensus.cc:2258] T 0889384793424e09ae21d8d5d84af957 P b61995e07c35456694b60e70309f2eef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:11.292696 27660 ts_tablet_manager.cc:1905] T 0889384793424e09ae21d8d5d84af957 P b61995e07c35456694b60e70309f2eef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:11.294451 27496 tablet_service.cc:1514] Processing DeleteTablet for tablet 0889384793424e09ae21d8d5d84af957 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:11 UTC) from {username='oryx', principal='oryx/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:47640
I20260807 15:10:11.294765 27658 tablet_replica.cc:330] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb: stopping tablet replica
I20260807 15:10:11.294904 27658 raft_consensus.cc:2229] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:10:11.295332 27658 raft_consensus.cc:2258] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:11.296229 27658 ts_tablet_manager.cc:1905] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:11.296303 27660 ts_tablet_manager.cc:1918] T 0889384793424e09ae21d8d5d84af957 P b61995e07c35456694b60e70309f2eef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:10:11.296391 27660 log.cc:1196] T 0889384793424e09ae21d8d5d84af957 P b61995e07c35456694b60e70309f2eef: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/0889384793424e09ae21d8d5d84af957
I20260807 15:10:11.296708 27660 ts_tablet_manager.cc:1939] T 0889384793424e09ae21d8d5d84af957 P b61995e07c35456694b60e70309f2eef: Deleting consensus metadata
I20260807 15:10:11.297663 26863 catalog_manager.cc:4841] TS b61995e07c35456694b60e70309f2eef (127.15.226.66:42627): tablet 0889384793424e09ae21d8d5d84af957 (table test-table [id=f73cffd73974498488fcb5dce858d90c]) successfully deleted
I20260807 15:10:11.298849 27658 ts_tablet_manager.cc:1918] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 15:10:11.298955 27658 log.cc:1196] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/0889384793424e09ae21d8d5d84af957
I20260807 15:10:11.299247 27658 ts_tablet_manager.cc:1939] T 0889384793424e09ae21d8d5d84af957 P 17aafb9028354f5ea9c4302d7d3c3dbb: Deleting consensus metadata
I20260807 15:10:11.300800 26862 catalog_manager.cc:4841] TS 17aafb9028354f5ea9c4302d7d3c3dbb (127.15.226.67:37875): tablet 0889384793424e09ae21d8d5d84af957 (table test-table [id=f73cffd73974498488fcb5dce858d90c]) successfully deleted
W20260807 15:10:11.311015 26863 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.226.65:38203 (error 108)
I20260807 15:10:11.311152 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 27195
I20260807 15:10:11.318500 27749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:11.317928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:38203 (local address 127.0.0.1:46560)
0807 15:10:11.318280 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:11.318375 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.65:38203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":80,"threads_started":1}
W20260807 15:10:11.319059 26863 catalog_manager.cc:4584] TS 0dc5606586144eb182d4d1459833528c (127.15.226.65:38203): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0889384793424e09ae21d8d5d84af957: Network error: Client connection negotiation failed: client connection to 127.15.226.65:38203: connect: Connection refused (error 111)
I20260807 15:10:11.329180 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 27396
I20260807 15:10:11.350826 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 26833
2026-08-07T15:10:11Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3886 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:10:11 dist-test-slave-3kk6 krb5kdc[27755](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:10:11 dist-test-slave-3kk6 krb5kdc[27755](info): set up 2 sockets
Aug 07 15:10:11 dist-test-slave-3kk6 krb5kdc[27755](info): commencing operation
krb5kdc: starting...
W20260807 15:10:12.635398 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:10:12 dist-test-slave-3kk6 krb5kdc[27755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:10:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:10:12.778404 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:44153
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:37629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:44153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:12.908308 27776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:12.908665 27776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:12.908753 27776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:12.913137 27776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:12.913259 27776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:12.913307 27776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:12.913363 27776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:12.913414 27776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:12.919209 27776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:37629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:44153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:44153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:12.920476 27776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:12.921628 27776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:12.931388 27790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:12.931352 27792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:12.931329 27787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:12.931869 27776 server_base.cc:1034] running on GCE node
I20260807 15:10:12.932950 27776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:12.934422 27776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:12.935638 27776 hybrid_clock.cc:648] HybridClock initialized: now 1786115412935646 us; error 71 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T15:10:12Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1584 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:10:13 dist-test-slave-3kk6 krb5kdc[27798](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:10:13 dist-test-slave-3kk6 krb5kdc[27798](info): set up 2 sockets
Aug 07 15:10:13 dist-test-slave-3kk6 krb5kdc[27798](info): commencing operation
krb5kdc: starting...
W20260807 15:10:14.210480 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:10:14 dist-test-slave-3kk6 krb5kdc[27798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:10:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:10:14.350335 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:34719
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:34719
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:14.491009 27821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:14.491410 27821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:14.491504 27821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:14.495774 27821 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:14.495898 27821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:14.495939 27821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:14.495981 27821 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:14.496013 27821 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:14.501958 27821 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:34719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:34719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:14.503280 27821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:14.504401 27821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:10:14.514034 27821 server_base.cc:1034] running on GCE node
W20260807 15:10:14.514026 27829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:14.514026 27826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:14.514273 27827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:14.514995 27821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:14.516160 27821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:14.517323 27821 hybrid_clock.cc:648] HybridClock initialized: now 1786115414517305 us; error 37 us; skew 500 ppm
Aug 07 15:10:14 dist-test-slave-3kk6 krb5kdc[27798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115414, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:14.520510 27821 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:10:14.521507 27821 webserver.cc:460] Webserver started at http://127.15.226.126:44887/ using document root <none> and password file <none>
I20260807 15:10:14.522063 27821 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:14.522125 27821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:14.522302 27821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:14.524552 27821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "4d02ed5bff654442ba6b6edb18113eeb"
format_stamp: "Formatted at 2026-08-07 15:10:14 on dist-test-slave-3kk6"
server_key: "821e8b9b3cb0492275a676a8caa41065"
server_key_iv: "c0cff852c9a8755bb320091e736b539c"
server_key_version: "encryptionkey@0"
I20260807 15:10:14.525121 27821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "4d02ed5bff654442ba6b6edb18113eeb"
format_stamp: "Formatted at 2026-08-07 15:10:14 on dist-test-slave-3kk6"
server_key: "821e8b9b3cb0492275a676a8caa41065"
server_key_iv: "c0cff852c9a8755bb320091e736b539c"
server_key_version: "encryptionkey@0"
I20260807 15:10:14.529414 27821 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260807 15:10:14.531987 27838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:14.533308 27821 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:10:14.533443 27821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "4d02ed5bff654442ba6b6edb18113eeb"
format_stamp: "Formatted at 2026-08-07 15:10:14 on dist-test-slave-3kk6"
server_key: "821e8b9b3cb0492275a676a8caa41065"
server_key_iv: "c0cff852c9a8755bb320091e736b539c"
server_key_version: "encryptionkey@0"
I20260807 15:10:14.533563 27821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:14.560453 27821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:14.564397 27821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:14.564635 27821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:14.572916 27821 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:34719
I20260807 15:10:14.572947 27918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:34719 every 8 connection(s)
I20260807 15:10:14.574057 27821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:14.577121 27921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:14.580200 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 27821
I20260807 15:10:14.580369 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:10:14.580760 16265 external_mini_cluster.cc:1428] Setting key a834a1b1169a63085f8c5c82e08e3a4f
I20260807 15:10:14.584014 27921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb: Bootstrap starting.
I20260807 15:10:14.586334 27921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:14.587364 27921 log.cc:824] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:14.589979 27921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb: No bootstrap required, opened a new log
Aug 07 15:10:14 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:10:14.593684 27921 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d02ed5bff654442ba6b6edb18113eeb" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34719 } }
I20260807 15:10:14.593961 27921 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:14.594027 27921 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d02ed5bff654442ba6b6edb18113eeb, State: Initialized, Role: FOLLOWER
I20260807 15:10:14.594511 27921 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d02ed5bff654442ba6b6edb18113eeb" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34719 } }
I20260807 15:10:14.594645 27921 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:14.594792 27921 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:14.594930 27921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:14.595983 27921 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d02ed5bff654442ba6b6edb18113eeb" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34719 } }
I20260807 15:10:14.596359 27921 leader_election.cc:302] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d02ed5bff654442ba6b6edb18113eeb; no voters:
I20260807 15:10:14.596751 27921 leader_election.cc:288] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:10:14.596885 27929 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:14.597131 27929 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [term 1 LEADER]: Becoming Leader. State: Replica: 4d02ed5bff654442ba6b6edb18113eeb, State: Running, Role: LEADER
I20260807 15:10:14.597525 27929 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d02ed5bff654442ba6b6edb18113eeb" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34719 } }
I20260807 15:10:14.598197 27921 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:14.598224 27925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:14.582371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57646 (local address 127.15.226.126:34719)
0807 15:10:14.582920 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:14.582935 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:14.582974 (+ 39us) server_negotiation.cc:407] Connection header received
0807 15:10:14.583613 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:14.583631 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:14.583992 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:14.584390 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:14.585283 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:14.586331 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:14.587671 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:14.588379 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:14.588515 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:14.591718 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:14.591724 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:14.591739 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:14.591769 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:14.594577 (+ 2808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:14.595265 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:14.595270 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:14.595275 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:14.595406 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:14.595713 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:14.595717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:14.595719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:14.596212 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:14.596439 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:14.596709 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 15:10:14.597198 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":179,"threads_started":1}
I20260807 15:10:14.600498 27930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d02ed5bff654442ba6b6edb18113eeb. Latest consensus state: current_term: 1 leader_uuid: "4d02ed5bff654442ba6b6edb18113eeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d02ed5bff654442ba6b6edb18113eeb" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34719 } } }
I20260807 15:10:14.600644 27930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:14.600745 27931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d02ed5bff654442ba6b6edb18113eeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d02ed5bff654442ba6b6edb18113eeb" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34719 } } }
I20260807 15:10:14.600894 27931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:14.601297 27953 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:14.604091 27953 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:10:14.610018 27953 catalog_manager.cc:1347] Generated new cluster ID: 5f84be8b9a164cc889ec531554f67bac
I20260807 15:10:14.610109 27953 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:10:14.628161 27953 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:10:14.629354 27953 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:14.640407 27953 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb: Generated new TSK 0
I20260807 15:10:14.641268 27953 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.226.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
I20260807 15:10:14.682163 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34719
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:14.810321 27964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:14.810626 27964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:14.810755 27964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:14.814980 27964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:14.815071 27964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:14.815156 27964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:10:14.821746 27964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:14.823365 27964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:14.824680 27964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:14.836423 27971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:14.836575 27972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:14.836423 27975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:14.836737 27964 server_base.cc:1034] running on GCE node
I20260807 15:10:14.837293 27964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:14.838081 27964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:14.839274 27964 hybrid_clock.cc:648] HybridClock initialized: now 1786115414839247 us; error 50 us; skew 500 ppm
Aug 07 15:10:14 dist-test-slave-3kk6 krb5kdc[27798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115414, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:14.842442 27964 init.cc:377] Logged in from keytab as kudu/127.15.226.65@KRBTEST.COM (short username kudu)
I20260807 15:10:14.843720 27964 webserver.cc:460] Webserver started at http://127.15.226.65:35509/ using document root <none> and password file <none>
I20260807 15:10:14.844441 27964 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:14.844512 27964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:14.844787 27964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:14.847126 27964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "a4b1b035ab5a450786e45f67bd352cbb"
format_stamp: "Formatted at 2026-08-07 15:10:14 on dist-test-slave-3kk6"
server_key: "2310ca79f370fb9dc1ec869d54f11e13"
server_key_iv: "a5982de2ba76ba331b01babbeef53c45"
server_key_version: "encryptionkey@0"
I20260807 15:10:14.847751 27964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "a4b1b035ab5a450786e45f67bd352cbb"
format_stamp: "Formatted at 2026-08-07 15:10:14 on dist-test-slave-3kk6"
server_key: "2310ca79f370fb9dc1ec869d54f11e13"
server_key_iv: "a5982de2ba76ba331b01babbeef53c45"
server_key_version: "encryptionkey@0"
I20260807 15:10:14.851802 27964 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:10:14.854367 27982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:14.855731 27964 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:10:14.855865 27964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "a4b1b035ab5a450786e45f67bd352cbb"
format_stamp: "Formatted at 2026-08-07 15:10:14 on dist-test-slave-3kk6"
server_key: "2310ca79f370fb9dc1ec869d54f11e13"
server_key_iv: "a5982de2ba76ba331b01babbeef53c45"
server_key_version: "encryptionkey@0"
I20260807 15:10:14.855986 27964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:14.869058 27964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:14.873160 27964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:14.873426 27964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:14.874140 27964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:14.875206 27964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:14.875263 27964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:14.875337 27964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:14.875373 27964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:14.885839 27964 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:36897
I20260807 15:10:14.885862 28142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:36897 every 8 connection(s)
I20260807 15:10:14.886960 27964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:10:14.891412 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 27964
I20260807 15:10:14.891588 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:10:14.892000 16265 external_mini_cluster.cc:1428] Setting key 093ae053d95ad1b7ebc6acb77edb3439
Aug 07 15:10:14 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115414, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.226.66@KRBTEST.COM; defaulting to no policy
I20260807 15:10:14.904115 27925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:14.889106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:47873 (local address 127.15.226.126:34719)
0807 15:10:14.889249 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:14.889253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:14.890151 (+ 898us) server_negotiation.cc:407] Connection header received
0807 15:10:14.891547 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:14.891552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:14.891621 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:14.891742 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:14.893023 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:14.893716 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:14.895716 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:14.896171 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:14.896316 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:14.899542 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:14.899545 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:14.899548 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:14.899576 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:14.901357 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:14.902116 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:14.902119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:14.902120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:14.902201 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:14.902644 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:14.902646 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:14.902697 (+ 51us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:14.902913 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:14.902995 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:14.903805 (+ 810us) server_negotiation.cc:299] Negotiation successful
0807 15:10:14.903928 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:10:14.905092 28146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:14.889466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.65:47873)
0807 15:10:14.889969 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:14.890016 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:14.891297 (+ 1281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:14.891904 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:14.891911 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:14.892364 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:14.892880 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:14.892894 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:14.893919 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:14.893926 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:14.895523 (+ 1597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:14.895541 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:14.896356 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:14.896359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:14.896531 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:14.897278 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:14.897298 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:14.899301 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:14.901513 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:14.901522 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:14.901545 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:14.901993 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:14.902299 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:14.902304 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:14.902308 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:14.902536 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:14.903038 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:14.903049 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:14.903474 (+ 425us) client_negotiation.cc:769] Sending connection context
0807 15:10:14.903723 (+ 249us) client_negotiation.cc:240] Negotiation successful
0807 15:10:14.904161 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":132,"threads_started":1}
I20260807 15:10:14.906455 28143 heartbeater.cc:347] Connected to a master server at 127.15.226.126:34719
I20260807 15:10:14.906749 28143 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.66@KRBTEST.COM" created.
I20260807 15:10:14.907334 28143 heartbeater.cc:511] Master 127.15.226.126:34719 requested a full tablet report, sending...
I20260807 15:10:14.909116 27874 ts_manager.cc:194] Registered new tserver with Master: a4b1b035ab5a450786e45f67bd352cbb (127.15.226.65:36897)
I20260807 15:10:14.910369 27874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.65@KRBTEST.COM'} at 127.15.226.65:47873
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
I20260807 15:10:14.927124 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34719
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:15.051995 28150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:15.052337 28150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:15.052384 28150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:15.056629 28150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:15.056713 28150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:15.056813 28150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:10:15.062737 28150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:15.063952 28150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:15.065120 28150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:15.075726 28166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:15.075726 28161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:15.075765 28164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:15.075989 28150 server_base.cc:1034] running on GCE node
I20260807 15:10:15.076571 28150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:15.077329 28150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:15.078543 28150 hybrid_clock.cc:648] HybridClock initialized: now 1786115415078522 us; error 34 us; skew 500 ppm
Aug 07 15:10:15 dist-test-slave-3kk6 krb5kdc[27798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115415, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:15.082058 28150 init.cc:377] Logged in from keytab as kudu/127.15.226.66@KRBTEST.COM (short username kudu)
I20260807 15:10:15.083098 28150 webserver.cc:460] Webserver started at http://127.15.226.66:33729/ using document root <none> and password file <none>
I20260807 15:10:15.083695 28150 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:15.083801 28150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:15.084043 28150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:15.086323 28150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "003f8f26b14c44c1a1b514aa4cb4a442"
format_stamp: "Formatted at 2026-08-07 15:10:15 on dist-test-slave-3kk6"
server_key: "ba9e1bbdd2dad5aa1b495b3ba1997615"
server_key_iv: "9ab7847057198fb07675e9c62485bd44"
server_key_version: "encryptionkey@0"
I20260807 15:10:15.087039 28150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "003f8f26b14c44c1a1b514aa4cb4a442"
format_stamp: "Formatted at 2026-08-07 15:10:15 on dist-test-slave-3kk6"
server_key: "ba9e1bbdd2dad5aa1b495b3ba1997615"
server_key_iv: "9ab7847057198fb07675e9c62485bd44"
server_key_version: "encryptionkey@0"
I20260807 15:10:15.091586 28150 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:10:15.094444 28175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:15.095960 28150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:10:15.096122 28150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "003f8f26b14c44c1a1b514aa4cb4a442"
format_stamp: "Formatted at 2026-08-07 15:10:15 on dist-test-slave-3kk6"
server_key: "ba9e1bbdd2dad5aa1b495b3ba1997615"
server_key_iv: "9ab7847057198fb07675e9c62485bd44"
server_key_version: "encryptionkey@0"
I20260807 15:10:15.096259 28150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:15.108116 28150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:15.111812 28150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:15.111987 28150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:15.112561 28150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:15.113412 28150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:15.113463 28150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:15.113507 28150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:15.113529 28150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:15.122993 28150 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:45865
I20260807 15:10:15.123025 28334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:45865 every 8 connection(s)
I20260807 15:10:15.124035 28150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:10:15.126542 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 28150
I20260807 15:10:15.126699 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:10:15.127060 16265 external_mini_cluster.cc:1428] Setting key 90b43197f8f0ff80316371118bb35c3f
Aug 07 15:10:15 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115415, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.226.67@KRBTEST.COM; defaulting to no policy
I20260807 15:10:15.141772 27925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:15.126053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:44455 (local address 127.15.226.126:34719)
0807 15:10:15.126202 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:15.126206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:15.127761 (+ 1555us) server_negotiation.cc:407] Connection header received
0807 15:10:15.128920 (+ 1159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:15.128923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:15.128968 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:15.129033 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:15.130848 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:15.131657 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:15.133386 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:15.133785 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:15.133930 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:15.137309 (+ 3379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:15.137312 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:15.137315 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:15.137343 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:15.139184 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:15.139898 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:15.139901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:15.139903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:15.139987 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:15.140409 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:15.140412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:15.140414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:15.140727 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:15.140853 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:15.141443 (+ 590us) server_negotiation.cc:299] Negotiation successful
0807 15:10:15.141587 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 15:10:15.142874 28339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:15.126401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.66:44455)
0807 15:10:15.127578 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:15.127624 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:15.128669 (+ 1045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:15.129262 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:15.129274 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:15.129855 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:15.130599 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:15.130623 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:15.131821 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:15.131827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:15.133224 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:15.133239 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:15.133965 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:15.133967 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:15.134127 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:15.134934 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:15.134960 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:15.137038 (+ 2078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:15.139316 (+ 2278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:15.139323 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:15.139341 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:15.139760 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:15.140083 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:15.140087 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:15.140090 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:15.140299 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:15.140843 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:15.140851 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:15.141172 (+ 321us) client_negotiation.cc:769] Sending connection context
0807 15:10:15.141383 (+ 211us) client_negotiation.cc:240] Negotiation successful
0807 15:10:15.141764 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":942,"thread_start_us":755,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.67@KRBTEST.COM" created.
I20260807 15:10:15.144246 28335 heartbeater.cc:347] Connected to a master server at 127.15.226.126:34719
I20260807 15:10:15.144537 28335 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:15.145140 28335 heartbeater.cc:511] Master 127.15.226.126:34719 requested a full tablet report, sending...
I20260807 15:10:15.146345 27874 ts_manager.cc:194] Registered new tserver with Master: 003f8f26b14c44c1a1b514aa4cb4a442 (127.15.226.66:45865)
I20260807 15:10:15.147063 27874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.66@KRBTEST.COM'} at 127.15.226.66:44455
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
I20260807 15:10:15.162752 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34719
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:15.285265 28342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:15.285559 28342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:15.285604 28342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:15.289685 28342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:15.289769 28342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:15.289832 28342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:10:15.295323 28342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:15.296519 28342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:15.297636 28342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:15.308182 28352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:15.308177 28355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:15.308261 28342 server_base.cc:1034] running on GCE node
W20260807 15:10:15.308169 28357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:15.308890 28342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:15.309566 28342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:15.310722 28342 hybrid_clock.cc:648] HybridClock initialized: now 1786115415310688 us; error 50 us; skew 500 ppm
Aug 07 15:10:15 dist-test-slave-3kk6 krb5kdc[27798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115415, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:15.313741 28342 init.cc:377] Logged in from keytab as kudu/127.15.226.67@KRBTEST.COM (short username kudu)
I20260807 15:10:15.314769 28342 webserver.cc:460] Webserver started at http://127.15.226.67:42447/ using document root <none> and password file <none>
I20260807 15:10:15.315407 28342 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:15.315501 28342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:15.315740 28342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:15.318009 28342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "dbd772a8134b4a4baa7895ee33ac98ff"
format_stamp: "Formatted at 2026-08-07 15:10:15 on dist-test-slave-3kk6"
server_key: "7e83866f6126b7be189ee34029fc3323"
server_key_iv: "210c579a1f78353c23bb980818ee39f3"
server_key_version: "encryptionkey@0"
I20260807 15:10:15.318620 28342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "dbd772a8134b4a4baa7895ee33ac98ff"
format_stamp: "Formatted at 2026-08-07 15:10:15 on dist-test-slave-3kk6"
server_key: "7e83866f6126b7be189ee34029fc3323"
server_key_iv: "210c579a1f78353c23bb980818ee39f3"
server_key_version: "encryptionkey@0"
I20260807 15:10:15.322862 28342 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:10:15.325640 28364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:15.327121 28342 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:10:15.327291 28342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "dbd772a8134b4a4baa7895ee33ac98ff"
format_stamp: "Formatted at 2026-08-07 15:10:15 on dist-test-slave-3kk6"
server_key: "7e83866f6126b7be189ee34029fc3323"
server_key_iv: "210c579a1f78353c23bb980818ee39f3"
server_key_version: "encryptionkey@0"
I20260807 15:10:15.327430 28342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:15.348161 28342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:15.351869 28342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:15.352097 28342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:15.352763 28342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:15.353636 28342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:15.353708 28342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:15.353777 28342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:15.353822 28342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:15.363984 28342 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:35573
I20260807 15:10:15.364005 28514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:35573 every 8 connection(s)
I20260807 15:10:15.365013 28342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:10:15.371559 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 28342
I20260807 15:10:15.371742 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:10:15.372068 16265 external_mini_cluster.cc:1428] Setting key 54a9ac454b0c9d9432b4c96a03d61909
Aug 07 15:10:15 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115415, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:10:15.381129 27925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:15.367271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:53983 (local address 127.15.226.126:34719)
0807 15:10:15.367473 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:15.367494 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:15.368409 (+ 915us) server_negotiation.cc:407] Connection header received
0807 15:10:15.369877 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:15.369881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:15.369946 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:15.370041 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:15.371531 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:15.372238 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:15.373717 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:15.373953 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:15.374067 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:15.376843 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:15.376847 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:15.376849 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:15.376878 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:15.378624 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:15.379251 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:15.379255 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:15.379256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:15.379357 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:15.379827 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:15.379830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:15.379832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:15.380077 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:15.380175 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:15.380863 (+ 688us) server_negotiation.cc:299] Negotiation successful
0807 15:10:15.380998 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 15:10:15.382226 28519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:15.367732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.67:53983)
0807 15:10:15.368243 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:15.368290 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:15.369632 (+ 1342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:15.370243 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:15.370251 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:15.370747 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:15.371364 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:15.371383 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:15.372376 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:15.372383 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:15.373588 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:15.373598 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:15.374116 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:15.374120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:15.374347 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:15.375062 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:15.375078 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:15.376654 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:15.378812 (+ 2158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:15.378817 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:15.378831 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:15.379118 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:15.379528 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:15.379531 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:15.379532 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:15.379692 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:15.380214 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:15.380221 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:15.380537 (+ 316us) client_negotiation.cc:769] Sending connection context
0807 15:10:15.380740 (+ 203us) client_negotiation.cc:240] Negotiation successful
0807 15:10:15.381145 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":129,"threads_started":1}
I20260807 15:10:15.383812 28515 heartbeater.cc:347] Connected to a master server at 127.15.226.126:34719
I20260807 15:10:15.384117 28515 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:15.384745 28515 heartbeater.cc:511] Master 127.15.226.126:34719 requested a full tablet report, sending...
I20260807 15:10:15.386019 27874 ts_manager.cc:194] Registered new tserver with Master: dbd772a8134b4a4baa7895ee33ac98ff (127.15.226.67:35573)
I20260807 15:10:15.386740 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:10:15.386972 27874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.67@KRBTEST.COM'} at 127.15.226.67:53983
WARNING: no policy specified for oryx/127.15.226.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.226.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.226.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.65.keytab.
Entry for principal oryx/127.15.226.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.65.keytab.
WARNING: no policy specified for oryx/127.15.226.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.226.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.226.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.66.keytab.
Entry for principal oryx/127.15.226.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.66.keytab.
WARNING: no policy specified for oryx/127.15.226.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.226.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.226.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.67.keytab.
Entry for principal oryx/127.15.226.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.67.keytab.
I20260807 15:10:15.481573 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 27964
I20260807 15:10:15.495450 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 28150
I20260807 15:10:15.510972 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 28342
I20260807 15:10:15.520699 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 27821
I20260807 15:10:15.534588 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:34719
--webserver_interface=127.15.226.126
--webserver_port=44887
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:34719
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:15.656291 28529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:15.656653 28529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:15.656699 28529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:15.660867 28529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:15.660964 28529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:15.660984 28529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:15.661005 28529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:15.661019 28529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:15.666612 28529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:34719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:34719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=44887
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:15.667882 28529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:15.668915 28529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:15.677951 28538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:15.677951 28535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:15.677970 28534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:15.678260 28529 server_base.cc:1034] running on GCE node
I20260807 15:10:15.678996 28529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:15.680148 28529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:15.681371 28529 hybrid_clock.cc:648] HybridClock initialized: now 1786115415681337 us; error 51 us; skew 500 ppm
Aug 07 15:10:15 dist-test-slave-3kk6 krb5kdc[27798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115415, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:15.684595 28529 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:10:15.685599 28529 webserver.cc:460] Webserver started at http://127.15.226.126:44887/ using document root <none> and password file <none>
I20260807 15:10:15.686203 28529 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:15.686318 28529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:15.690488 28529 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:10:15.692934 28547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:15.694371 28529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:10:15.694541 28529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "4d02ed5bff654442ba6b6edb18113eeb"
format_stamp: "Formatted at 2026-08-07 15:10:14 on dist-test-slave-3kk6"
server_key: "821e8b9b3cb0492275a676a8caa41065"
server_key_iv: "c0cff852c9a8755bb320091e736b539c"
server_key_version: "encryptionkey@0"
I20260807 15:10:15.695238 28529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:15.713913 28529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:15.717666 28529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:15.717909 28529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:15.726032 28529 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:34719
I20260807 15:10:15.726058 28620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:34719 every 8 connection(s)
I20260807 15:10:15.727308 28529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:15.731911 28621 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:10:15.733168 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 28529
I20260807 15:10:15.733716 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:36897
--local_ip_for_outbound_sockets=127.15.226.65
--tserver_master_addrs=127.15.226.126:34719
--webserver_port=35509
--webserver_interface=127.15.226.65
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.226.65
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:10:15.734048 28621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb: Bootstrap starting.
I20260807 15:10:15.753932 28621 log.cc:824] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:15.761082 28621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:10:15.761518 28621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb: Bootstrap complete.
I20260807 15:10:15.765998 28621 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d02ed5bff654442ba6b6edb18113eeb" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34719 } }
I20260807 15:10:15.766440 28621 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d02ed5bff654442ba6b6edb18113eeb, State: Initialized, Role: FOLLOWER
I20260807 15:10:15.767083 28621 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d02ed5bff654442ba6b6edb18113eeb" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34719 } }
I20260807 15:10:15.767216 28621 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:15.767285 28621 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:15.767385 28621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:10:15.769541 28621 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d02ed5bff654442ba6b6edb18113eeb" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34719 } }
I20260807 15:10:15.769886 28621 leader_election.cc:302] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d02ed5bff654442ba6b6edb18113eeb; no voters:
I20260807 15:10:15.770248 28621 leader_election.cc:288] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 15:10:15.770346 28625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:10:15.770576 28625 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [term 2 LEADER]: Becoming Leader. State: Replica: 4d02ed5bff654442ba6b6edb18113eeb, State: Running, Role: LEADER
I20260807 15:10:15.770913 28625 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d02ed5bff654442ba6b6edb18113eeb" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34719 } }
I20260807 15:10:15.771616 28621 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:15.772441 28627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4d02ed5bff654442ba6b6edb18113eeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d02ed5bff654442ba6b6edb18113eeb" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34719 } } }
I20260807 15:10:15.772588 28627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:15.773276 28640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:15.773243 28629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d02ed5bff654442ba6b6edb18113eeb. Latest consensus state: current_term: 2 leader_uuid: "4d02ed5bff654442ba6b6edb18113eeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d02ed5bff654442ba6b6edb18113eeb" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34719 } } }
I20260807 15:10:15.773464 28629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:15.777010 28640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:10:15.778026 28640 catalog_manager.cc:1259] Loaded cluster ID: 5f84be8b9a164cc889ec531554f67bac
I20260807 15:10:15.778101 28640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:10:15.780575 28640 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:15.781268 28640 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4d02ed5bff654442ba6b6edb18113eeb: Loaded TSK: 0
I20260807 15:10:15.782548 28640 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:10:15.858469 28623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:15.858834 28623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:15.858887 28623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:15.863055 28623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:15.863124 28623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:15.863186 28623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:10:15.868729 28623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.65.keytab
--principal=oryx/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:36897
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=35509
--tserver_master_addrs=127.15.226.126:34719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:15.869807 28623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:15.870832 28623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:15.881846 28653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:15.881831 28656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:15.881825 28654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:15.882428 28623 server_base.cc:1034] running on GCE node
I20260807 15:10:15.882958 28623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:15.883754 28623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:15.885027 28623 hybrid_clock.cc:648] HybridClock initialized: now 1786115415884989 us; error 67 us; skew 500 ppm
Aug 07 15:10:15 dist-test-slave-3kk6 krb5kdc[27798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115415, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:15.888446 28623 init.cc:377] Logged in from keytab as oryx/127.15.226.65@KRBTEST.COM (short username oryx)
I20260807 15:10:15.889583 28623 webserver.cc:460] Webserver started at http://127.15.226.65:35509/ using document root <none> and password file <none>
I20260807 15:10:15.890182 28623 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:15.890296 28623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:15.894474 28623 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:10:15.896790 28664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:15.898113 28623 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:10:15.898250 28623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "a4b1b035ab5a450786e45f67bd352cbb"
format_stamp: "Formatted at 2026-08-07 15:10:14 on dist-test-slave-3kk6"
server_key: "2310ca79f370fb9dc1ec869d54f11e13"
server_key_iv: "a5982de2ba76ba331b01babbeef53c45"
server_key_version: "encryptionkey@0"
I20260807 15:10:15.898942 28623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:15.928454 28623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:15.932345 28623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:15.932576 28623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:15.933243 28623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:15.934167 28623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:15.934239 28623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:15.934324 28623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:15.934373 28623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:15.944284 28623 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:36897
I20260807 15:10:15.944406 28814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:36897 every 8 connection(s)
I20260807 15:10:15.945473 28623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:10:15.953209 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 28623
I20260807 15:10:15.953758 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:45865
--local_ip_for_outbound_sockets=127.15.226.66
--tserver_master_addrs=127.15.226.126:34719
--webserver_port=33729
--webserver_interface=127.15.226.66
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.226.66
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:10:15 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:15 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:15.960896 28819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:15.948108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.65:44931)
0807 15:10:15.949082 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:15.949127 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:15.949939 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:15.951220 (+ 1281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:15.951227 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:15.951610 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:15.952139 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:15.952154 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:15.953682 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:15.953690 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:15.955269 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:15.955284 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:15.956164 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:15.956171 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:15.956411 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:15.957310 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:15.957327 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:15.960093 (+ 2766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":765,"thread_start_us":176,"threads_started":1}
W20260807 15:10:15.961386 28815 heartbeater.cc:644] Failed to heartbeat to 127.15.226.126:34719 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.226.126:34719: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
W20260807 15:10:15.961424 28817 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:15.947997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:44931 (local address 127.15.226.126:34719)
0807 15:10:15.948503 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:15.948523 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:15.949281 (+ 758us) server_negotiation.cc:407] Connection header received
0807 15:10:15.950372 (+ 1091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:15.950384 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:15.950809 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:15.951113 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:15.952355 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:15.953522 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:15.955431 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:15.955966 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:15.956106 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:15.960447 (+ 4341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.65:44931: BlockingRecv error: recv got EOF from 127.15.226.65:44931 (error 108)
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":118,"threads_started":1}
Aug 07 15:10:15 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:15 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:15.968112 28819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:15.961926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.65:36373)
0807 15:10:15.962169 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:15.962181 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:15.962258 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:15.962534 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:15.962536 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:15.962719 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:15.962815 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:15.962820 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:15.963959 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:15.963962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:15.964523 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:15.964531 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:15.965212 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:15.965215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:15.965308 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:15.965868 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:15.965879 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:15.968008 (+ 2129us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184}
W20260807 15:10:15.968314 28817 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:15.961988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:36373 (local address 127.15.226.126:34719)
0807 15:10:15.962098 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:15.962101 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:15.962253 (+ 152us) server_negotiation.cc:407] Connection header received
0807 15:10:15.962343 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:15.962349 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:15.962417 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:15.962523 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:15.962933 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:15.963824 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:15.964649 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:15.965069 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:15.965216 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:15.968187 (+ 2971us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.65:36373: BlockingRecv error: recv got EOF from 127.15.226.65:36373 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 07 15:10:15 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:15 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:15.974303 28819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:15.968744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.65:56465)
0807 15:10:15.968956 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:15.968973 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:15.969078 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:15.969422 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:15.969427 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:15.969570 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:15.969643 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:15.969648 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:15.970386 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:15.970391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:15.970991 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:15.971000 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:15.971502 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:15.971506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:15.971584 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:15.972136 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:15.972148 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:15.974151 (+ 2003us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140}
W20260807 15:10:15.974431 28817 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:15.968824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:56465 (local address 127.15.226.126:34719)
0807 15:10:15.968943 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:15.968947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:15.969020 (+ 73us) server_negotiation.cc:407] Connection header received
0807 15:10:15.969182 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:15.969186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:15.969230 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:15.969342 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:15.969754 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:15.970267 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:15.971109 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:15.971389 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:15.971498 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:15.974285 (+ 2787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.65:56465: BlockingRecv error: recv got EOF from 127.15.226.65:56465 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260807 15:10:15.974624 28815 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:10:16.082155 28821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:16.082471 28821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:16.082523 28821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:16.087265 28821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:16.087349 28821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:16.087460 28821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:10:16.093691 28821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.66.keytab
--principal=oryx/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:45865
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=33729
--tserver_master_addrs=127.15.226.126:34719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:16.095073 28821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:16.096730 28821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:16.109975 28830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:16.109974 28829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:16.110039 28821 server_base.cc:1034] running on GCE node
W20260807 15:10:16.109974 28834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:16.110819 28821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:16.111558 28821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:16.112732 28821 hybrid_clock.cc:648] HybridClock initialized: now 1786115416112702 us; error 48 us; skew 500 ppm
Aug 07 15:10:16 dist-test-slave-3kk6 krb5kdc[27798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115416, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:16.115892 28821 init.cc:377] Logged in from keytab as oryx/127.15.226.66@KRBTEST.COM (short username oryx)
I20260807 15:10:16.116926 28821 webserver.cc:460] Webserver started at http://127.15.226.66:33729/ using document root <none> and password file <none>
I20260807 15:10:16.117507 28821 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:16.117595 28821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:16.122097 28821 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:10:16.124536 28843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:16.125913 28821 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:10:16.126036 28821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "003f8f26b14c44c1a1b514aa4cb4a442"
format_stamp: "Formatted at 2026-08-07 15:10:15 on dist-test-slave-3kk6"
server_key: "ba9e1bbdd2dad5aa1b495b3ba1997615"
server_key_iv: "9ab7847057198fb07675e9c62485bd44"
server_key_version: "encryptionkey@0"
I20260807 15:10:16.126708 28821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:16.142285 28821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:16.145972 28821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:16.146157 28821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:16.146827 28821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:16.147689 28821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:16.147756 28821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:16.147799 28821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:16.147815 28821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:16.157706 28821 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:45865
I20260807 15:10:16.157734 28998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:45865 every 8 connection(s)
I20260807 15:10:16.159015 28821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:10:16.163081 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 28821
I20260807 15:10:16.163626 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:35573
--local_ip_for_outbound_sockets=127.15.226.67
--tserver_master_addrs=127.15.226.126:34719
--webserver_port=42447
--webserver_interface=127.15.226.67
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.226.67
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:10:16 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:16 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 15:10:16.173274 28817 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:16.161092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:53511 (local address 127.15.226.126:34719)
0807 15:10:16.161281 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:16.161286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:16.162212 (+ 926us) server_negotiation.cc:407] Connection header received
0807 15:10:16.164045 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:16.164049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:16.164126 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:16.164220 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:16.166014 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.166991 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.168361 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.168829 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.168972 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:16.173154 (+ 4182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.66:53511: BlockingRecv error: recv got EOF from 127.15.226.66:53511 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 15:10:16.174373 29002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:16.161539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.66:53511)
0807 15:10:16.162034 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:16.162082 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:16.163761 (+ 1679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:16.164461 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:16.164474 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:16.165047 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:16.165816 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:16.165833 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.167147 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.167150 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:16.168213 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:16.168225 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.169037 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.169044 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:16.169220 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:16.169939 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:16.169958 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:16.173021 (+ 3063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":139,"threads_started":1}
W20260807 15:10:16.175014 28999 heartbeater.cc:644] Failed to heartbeat to 127.15.226.126:34719 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.226.126:34719: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Aug 07 15:10:16 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:16 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:16.181231 29002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:16.175544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.66:39701)
0807 15:10:16.175667 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:16.175685 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:16.175812 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:16.176105 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:16.176108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:16.176295 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:16.176400 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:16.176406 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.177167 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.177170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:16.177727 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:16.177735 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.178200 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.178203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:16.178299 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:16.178875 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:16.178888 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:16.181071 (+ 2183us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 15:10:16.181330 28817 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:16.175708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:39701 (local address 127.15.226.126:34719)
0807 15:10:16.175869 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:16.175875 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:16.175892 (+ 17us) server_negotiation.cc:407] Connection header received
0807 15:10:16.175952 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:16.175954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:16.175994 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:16.176082 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:16.176525 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.177053 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.177857 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.178077 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.178182 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:16.181206 (+ 3024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.66:39701: BlockingRecv error: recv got EOF from 127.15.226.66:39701 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 15:10:16 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:16 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:16.187678 29002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:16.181913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.66:44743)
0807 15:10:16.182067 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:16.182086 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:16.182172 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:16.182423 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:16.182426 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:16.182622 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:16.182781 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:16.182791 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.183506 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.183509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:16.184401 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:16.184411 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.184822 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.184825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:16.184910 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:16.185393 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:16.185407 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:16.187524 (+ 2117us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":69}
W20260807 15:10:16.187934 28817 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:16.182032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:44743 (local address 127.15.226.126:34719)
0807 15:10:16.182137 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:16.182140 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:16.182153 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:10:16.182269 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:16.182271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:16.182310 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:16.182388 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:16.182905 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.183396 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.184515 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.184718 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.184796 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:16.187661 (+ 2865us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.66:44743: BlockingRecv error: recv got EOF from 127.15.226.66:44743 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260807 15:10:16.187932 28999 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:10:16.288807 29004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:16.289139 29004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:16.289186 29004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:16.293502 29004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:16.293584 29004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:16.293648 29004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:10:16.299348 29004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/krb5kdc/oryx_127.15.226.67.keytab
--principal=oryx/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:35573
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=42447
--tserver_master_addrs=127.15.226.126:34719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:16.300567 29004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:16.301632 29004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:16.312338 29010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:16.312328 29014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:16.312328 29011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:16.312623 29004 server_base.cc:1034] running on GCE node
I20260807 15:10:16.313140 29004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:16.313846 29004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:16.315043 29004 hybrid_clock.cc:648] HybridClock initialized: now 1786115416315028 us; error 28 us; skew 500 ppm
Aug 07 15:10:16 dist-test-slave-3kk6 krb5kdc[27798](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115416, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:16.318106 29004 init.cc:377] Logged in from keytab as oryx/127.15.226.67@KRBTEST.COM (short username oryx)
I20260807 15:10:16.319165 29004 webserver.cc:460] Webserver started at http://127.15.226.67:42447/ using document root <none> and password file <none>
I20260807 15:10:16.319711 29004 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:16.319792 29004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:16.324039 29004 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260807 15:10:16.326328 29024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:16.327816 29004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 15:10:16.327971 29004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "dbd772a8134b4a4baa7895ee33ac98ff"
format_stamp: "Formatted at 2026-08-07 15:10:15 on dist-test-slave-3kk6"
server_key: "7e83866f6126b7be189ee34029fc3323"
server_key_iv: "210c579a1f78353c23bb980818ee39f3"
server_key_version: "encryptionkey@0"
I20260807 15:10:16.328593 29004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:16.366338 29004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:16.369975 29004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:16.370206 29004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:16.370890 29004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:16.371888 29004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:16.371963 29004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:16.372040 29004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:16.372082 29004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:16.382391 29004 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:35573
I20260807 15:10:16.382421 29187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:35573 every 8 connection(s)
I20260807 15:10:16.383620 29004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:10:16.393285 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 29004
Aug 07 15:10:16 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:16 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 15:10:16.396274 28817 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:16.385643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:44585 (local address 127.15.226.126:34719)
0807 15:10:16.385845 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:16.385849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:16.386728 (+ 879us) server_negotiation.cc:407] Connection header received
0807 15:10:16.388127 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:16.388130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:16.388181 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:16.388272 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:16.389738 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.390285 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.391669 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.391930 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.392033 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:16.396132 (+ 4099us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.67:44585: BlockingRecv error: recv got EOF from 127.15.226.67:44585 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 15:10:16.396896 29191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:16.386022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.67:44585)
0807 15:10:16.386526 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:16.386575 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:16.387885 (+ 1310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:16.388499 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:16.388507 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:16.388980 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:16.389524 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:16.389539 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.390418 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.390420 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:16.391524 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:16.391536 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.392103 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.392107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:16.392267 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:16.392930 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:16.392949 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:16.395953 (+ 3004us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":118,"threads_started":1}
W20260807 15:10:16.397583 29188 heartbeater.cc:644] Failed to heartbeat to 127.15.226.126:34719 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.226.126:34719: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Aug 07 15:10:16 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:16 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:16.404076 29191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:16.398274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.67:56645)
0807 15:10:16.398414 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:16.398430 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:16.398539 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:16.398830 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:16.398851 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:16.399025 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:16.399130 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:16.399136 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.399944 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.399955 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:16.400816 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:16.400826 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.401301 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.401305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:16.401391 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:16.401756 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:16.401770 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:16.403928 (+ 2158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 15:10:16.404167 28817 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:16.398367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:56645 (local address 127.15.226.126:34719)
0807 15:10:16.398484 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:16.398488 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:16.398501 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:10:16.398641 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:16.398644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:16.398710 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:16.398791 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:16.399273 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.399839 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.400968 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.401192 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.401300 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:16.404068 (+ 2768us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.67:56645: BlockingRecv error: recv got EOF from 127.15.226.67:56645 (error 108)
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":44}
I20260807 15:10:16.405507 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:16.394591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57652 (local address 127.15.226.126:34719)
0807 15:10:16.394873 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:16.394877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:16.394889 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:10:16.394926 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:16.394928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:16.394969 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:16.395052 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:16.396530 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.397102 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.398284 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.398632 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.398785 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:16.400315 (+ 1530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:16.400322 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:16.400341 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:16.400373 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:16.402639 (+ 2266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:16.403218 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:16.403229 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:16.403235 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:16.403370 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:16.403950 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:16.403954 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:16.403956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:16.404487 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:16.404697 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:16.405052 (+ 355us) server_negotiation.cc:299] Negotiation successful
0807 15:10:16.405328 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":205,"thread_start_us":122,"threads_started":1}
Aug 07 15:10:16 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:16 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:16.410202 29191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:16.404888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.67:45343)
0807 15:10:16.405050 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:16.405065 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:16.405136 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:16.405421 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:16.405424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:16.405588 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:16.405687 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:16.405693 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.406416 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.406421 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:16.407027 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:16.407036 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.407519 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.407523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:16.407599 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:16.408142 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:16.408156 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:16.410053 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98}
W20260807 15:10:16.410290 28817 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:16.404964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:45343 (local address 127.15.226.126:34719)
0807 15:10:16.405115 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:16.405119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:16.405134 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:10:16.405231 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:16.405235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:16.405300 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:16.405377 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:16.405811 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.406309 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.407160 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.407407 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.407506 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:16.410165 (+ 2659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.67:45343: BlockingRecv error: recv got EOF from 127.15.226.67:45343 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260807 15:10:16.410477 29188 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 15:10:16 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:16 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:16.981798 29196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:16.975386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.65:44819)
0807 15:10:16.975704 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:16.975725 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:16.975864 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:16.976114 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:16.976116 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:16.976319 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:16.976452 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:16.976457 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.977394 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.977396 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:16.977982 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:16.977990 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.978525 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.978528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:16.978608 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:16.979256 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:16.979280 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:16.981676 (+ 2396us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":84,"threads_started":1}
W20260807 15:10:16.982012 29197 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:16.975517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:44819 (local address 127.15.226.126:34719)
0807 15:10:16.975862 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:16.975866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:16.975882 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:10:16.975946 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:16.975949 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:16.975989 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:16.976072 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:16.976615 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.977280 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.978116 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:16.978400 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:16.978501 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:16.981830 (+ 3329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.65:44819: BlockingRecv error: recv got EOF from 127.15.226.65:44819 (error 108)
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":122,"threads_started":1}
Aug 07 15:10:17 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:17 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:17.194619 29198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:17.188694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.66:33467)
0807 15:10:17.189013 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:17.189028 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:17.189180 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:17.189519 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:17.189523 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:17.189805 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:17.189938 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:17.189947 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:17.190753 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:17.190756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:17.191348 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:17.191357 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:17.191837 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:17.191840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:17.191916 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:17.192506 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:17.192518 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:17.194483 (+ 1965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":118,"threads_started":1}
W20260807 15:10:17.194895 29197 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:17.188737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:33467 (local address 127.15.226.126:34719)
0807 15:10:17.188953 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:17.188959 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:17.189131 (+ 172us) server_negotiation.cc:407] Connection header received
0807 15:10:17.189283 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:17.189285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:17.189366 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:17.189467 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:17.190064 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:17.190603 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:17.191484 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:17.191726 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:17.191822 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:17.194756 (+ 2934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.66:33467: BlockingRecv error: recv got EOF from 127.15.226.66:33467 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
Aug 07 15:10:17 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:17 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:17.417200 29199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:17.411259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.67:49869)
0807 15:10:17.411569 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:17.411584 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:17.411703 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:17.412011 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:17.412014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:17.412210 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:17.412341 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:17.412347 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:17.413126 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:17.413129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:17.413715 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:17.413723 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:17.414177 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:17.414180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:17.414259 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:17.414947 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:17.414959 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:17.417014 (+ 2055us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":97,"threads_started":1}
W20260807 15:10:17.417312 29197 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:17.411353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:49869 (local address 127.15.226.126:34719)
0807 15:10:17.411522 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:17.411526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:17.411661 (+ 135us) server_negotiation.cc:407] Connection header received
0807 15:10:17.411791 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:17.411793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:17.411873 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:17.411948 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:17.412455 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:17.413010 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:17.413847 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:17.414064 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:17.414141 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:17.417179 (+ 3038us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.67:49869: BlockingRecv error: recv got EOF from 127.15.226.67:49869 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 15:10:17 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:17 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:17.993458 29201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:17.982886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.65:45729)
0807 15:10:17.983197 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:17.983216 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:17.983352 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:17.983656 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:17.983660 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:17.983966 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:17.984079 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:17.984086 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:17.984900 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:17.984905 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:17.985482 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:17.985490 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:17.985992 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:17.985996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:17.986090 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:17.991118 (+ 5028us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:17.991135 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:17.993257 (+ 2122us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":101,"threads_started":1}
W20260807 15:10:17.993604 29204 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:17.982902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:45729 (local address 127.15.226.126:34719)
0807 15:10:17.983202 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:17.983206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:17.983306 (+ 100us) server_negotiation.cc:407] Connection header received
0807 15:10:17.983456 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:17.983461 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:17.983519 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:17.983627 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:17.984222 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:17.984778 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:17.985612 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:17.985861 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:17.985972 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:17.993470 (+ 7498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.65:45729: BlockingRecv error: recv got EOF from 127.15.226.65:45729 (error 108)
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":105,"threads_started":1}
Aug 07 15:10:18 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:18 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:18.202489 29205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:18.195787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.66:34709)
0807 15:10:18.196140 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:18.196157 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:18.196285 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:18.196572 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:18.196575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:18.196833 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:18.196947 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:18.196953 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:18.197794 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:18.197797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:18.198498 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:18.198508 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:18.199179 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:18.199182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:18.199268 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:18.199921 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:18.199934 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:18.202279 (+ 2345us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":118,"threads_started":1}
W20260807 15:10:18.202525 29204 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:18.195923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:34709 (local address 127.15.226.126:34719)
0807 15:10:18.196145 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:18.196149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:18.196280 (+ 131us) server_negotiation.cc:407] Connection header received
0807 15:10:18.196367 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:18.196370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:18.196420 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:18.196503 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:18.197078 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:18.197618 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:18.198638 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:18.199011 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:18.199117 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:18.202397 (+ 3280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.66:34709: BlockingRecv error: recv got EOF from 127.15.226.66:34709 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 07 15:10:18 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:18 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:18.424023 29206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:18.418165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.67:54017)
0807 15:10:18.418478 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:18.418494 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:18.418594 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:18.418891 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:18.418893 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:18.419085 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:18.419186 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:18.419190 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:18.419915 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:18.419918 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:18.420470 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:18.420478 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:18.420901 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:18.420904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:18.420965 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:18.421580 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:18.421594 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:18.423900 (+ 2306us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":83,"threads_started":1}
W20260807 15:10:18.424203 29204 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:18.418359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:54017 (local address 127.15.226.126:34719)
0807 15:10:18.418527 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:18.418531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:18.418546 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:10:18.418683 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:18.418685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:18.418729 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:18.418819 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:18.419288 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:18.419815 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:18.420574 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:18.420794 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:18.420877 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:18.424043 (+ 3166us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.67:54017: BlockingRecv error: recv got EOF from 127.15.226.67:54017 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 07 15:10:18 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:18 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:19.000890 29207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:18.994551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.65:54555)
0807 15:10:18.994880 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:18.994902 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:18.995043 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:18.995328 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:18.995331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:18.995537 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:18.995643 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:18.995649 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:18.996462 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:18.996465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:18.997060 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:18.997069 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:18.997561 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:18.997564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:18.997627 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:18.998304 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:18.998316 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:19.000676 (+ 2360us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":107,"threads_started":1}
W20260807 15:10:19.000998 29208 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:18.994734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:54555 (local address 127.15.226.126:34719)
0807 15:10:18.995040 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:18.995044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:18.995070 (+ 26us) server_negotiation.cc:407] Connection header received
0807 15:10:18.995117 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:18.995119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:18.995162 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:18.995269 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:18.995794 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:18.996348 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:18.997195 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:18.997447 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:18.997542 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:19.000822 (+ 3280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.65:54555: BlockingRecv error: recv got EOF from 127.15.226.65:54555 (error 108)
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":95,"threads_started":1}
Aug 07 15:10:19 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:19 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:19.209079 29209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:19.203603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.66:44363)
0807 15:10:19.203889 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:19.203928 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:19.204066 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:19.204313 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:19.204315 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:19.204510 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:19.204611 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:19.204616 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:19.205388 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:19.205391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:19.205953 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:19.205961 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:19.206373 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:19.206376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:19.206440 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:19.207002 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:19.207015 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:19.208955 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":92,"threads_started":1}
W20260807 15:10:19.209380 29208 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:19.203657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:44363 (local address 127.15.226.126:34719)
0807 15:10:19.203827 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:19.203834 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:19.204040 (+ 206us) server_negotiation.cc:407] Connection header received
0807 15:10:19.204158 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:19.204160 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:19.204207 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:19.204279 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:19.204708 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:19.205290 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:19.206054 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:19.206274 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:19.206354 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:19.209127 (+ 2773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.66:44363: BlockingRecv error: recv got EOF from 127.15.226.66:44363 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 15:10:19 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:19 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:19.431350 29210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:19.424985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.67:51029)
0807 15:10:19.425284 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:19.425299 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:19.425412 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:19.425671 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:19.425674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:19.425888 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:19.425990 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:19.425997 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:19.426867 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:19.426870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:19.427466 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:19.427474 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:19.427911 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:19.427914 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:19.427980 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:19.428656 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:19.428675 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:19.431140 (+ 2465us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":105,"threads_started":1}
W20260807 15:10:19.431742 29208 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:19.425055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:51029 (local address 127.15.226.126:34719)
0807 15:10:19.425210 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:19.425214 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:19.425377 (+ 163us) server_negotiation.cc:407] Connection header received
0807 15:10:19.425496 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:19.425498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:19.425549 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:19.425628 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:19.426131 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:19.426760 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:19.427568 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:19.427801 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:19.427886 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:19.431524 (+ 3638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.67:51029: BlockingRecv error: recv got EOF from 127.15.226.67:51029 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 07 15:10:20 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:20 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:20.007882 29211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:20.001884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.65:49627)
0807 15:10:20.002381 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:20.002398 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:20.002520 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:20.002893 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:20.002895 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:20.003102 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:20.003212 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:20.003217 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:20.004087 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:20.004090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:20.004653 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:20.004662 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:20.005102 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:20.005104 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:20.005168 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:20.005714 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:20.005727 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:20.007727 (+ 2000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":106,"threads_started":1}
W20260807 15:10:20.008052 29212 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:20.002136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:49627 (local address 127.15.226.126:34719)
0807 15:10:20.002402 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:20.002406 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:20.002483 (+ 77us) server_negotiation.cc:407] Connection header received
0807 15:10:20.002676 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:20.002679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:20.002732 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:20.002821 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:20.003325 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:20.003971 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:20.004754 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:20.005000 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:20.005080 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:20.007891 (+ 2811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.65:49627: BlockingRecv error: recv got EOF from 127.15.226.65:49627 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":69,"threads_started":1}
Aug 07 15:10:20 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:20 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:20.216966 29213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:20.210248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.66:59299)
0807 15:10:20.210608 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:20.210625 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:20.210767 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:20.211159 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:20.211162 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:20.211365 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:20.211485 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:20.211491 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:20.212370 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:20.212373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:20.212953 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:20.212962 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:20.213528 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:20.213531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:20.213607 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:20.214248 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:20.214262 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:20.216787 (+ 2525us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":125,"threads_started":1}
W20260807 15:10:20.217079 29212 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:20.210442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:59299 (local address 127.15.226.126:34719)
0807 15:10:20.210640 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:20.210644 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:20.210717 (+ 73us) server_negotiation.cc:407] Connection header received
0807 15:10:20.210931 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:20.210934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:20.210991 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:20.211081 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:20.211635 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:20.212258 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:20.213088 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:20.213346 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:20.213468 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:20.216957 (+ 3489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.66:59299: BlockingRecv error: recv got EOF from 127.15.226.66:59299 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 07 15:10:20 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:20 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:20.439297 29214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:20.432440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.67:43825)
0807 15:10:20.432894 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:20.432914 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:20.433055 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:20.433411 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:20.433414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:20.433700 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:20.433839 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:20.433847 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:20.435201 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:20.435205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:20.436060 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:20.436071 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:20.436551 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:20.436555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:20.436642 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:20.437188 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:20.437199 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:20.439116 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":126,"threads_started":1}
W20260807 15:10:20.439380 29212 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:20.432566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:43825 (local address 127.15.226.126:34719)
0807 15:10:20.432734 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:20.432739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:20.433021 (+ 282us) server_negotiation.cc:407] Connection header received
0807 15:10:20.433193 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:20.433197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:20.433267 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:20.433369 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:20.434012 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:20.435098 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:20.436183 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:20.436433 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:20.436544 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:20.439262 (+ 2718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.67:43825: BlockingRecv error: recv got EOF from 127.15.226.67:43825 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 15:10:21 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:21 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:21.015129 29216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:21.008989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.65:47229)
0807 15:10:21.009321 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:21.009337 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:21.009436 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:21.009702 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:21.009703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:21.009933 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:21.010038 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:21.010043 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:21.010877 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:21.010880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:21.011492 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:21.011501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:21.012036 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:21.012038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:21.012135 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:21.012882 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:21.012900 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:21.015004 (+ 2104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":109,"threads_started":1}
W20260807 15:10:21.015374 29217 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:21.009156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:47229 (local address 127.15.226.126:34719)
0807 15:10:21.009405 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:21.009409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:21.009425 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:10:21.009533 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:21.009536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:21.009580 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:21.009667 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:21.010167 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:21.010753 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:21.011622 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:21.011910 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:21.012035 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:21.015194 (+ 3159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.65:47229: BlockingRecv error: recv got EOF from 127.15.226.65:47229 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":70,"threads_started":1}
Aug 07 15:10:21 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:21 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:21.224646 29218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:21.218127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.66:42855)
0807 15:10:21.218507 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:21.218525 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:21.218710 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:21.219052 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:21.219055 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:21.219339 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:21.219447 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:21.219454 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:21.220411 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:21.220414 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:21.221114 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:21.221124 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:21.221659 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:21.221662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:21.221735 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:21.222415 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:21.222429 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:21.224516 (+ 2087us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":158,"threads_started":1}
W20260807 15:10:21.225028 29217 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:21.218379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:42855 (local address 127.15.226.126:34719)
0807 15:10:21.218526 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:21.218531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:21.218628 (+ 97us) server_negotiation.cc:407] Connection header received
0807 15:10:21.218824 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:21.218828 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:21.218910 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:21.219023 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:21.219577 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:21.220296 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:21.221259 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:21.221514 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:21.221608 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:21.224797 (+ 3189us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.66:42855: BlockingRecv error: recv got EOF from 127.15.226.66:42855 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 15:10:21 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:21 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:21.446463 29219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:21.440320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.67:42163)
0807 15:10:21.440629 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:21.440645 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:21.440755 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:21.441054 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:21.441057 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:21.441280 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:21.441391 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:21.441397 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:21.442274 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:21.442277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:21.443013 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:21.443026 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:21.443573 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:21.443576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:21.443653 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:21.444339 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:21.444352 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:21.446328 (+ 1976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":132,"threads_started":1}
W20260807 15:10:21.446731 29217 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:21.440407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:42163 (local address 127.15.226.126:34719)
0807 15:10:21.440564 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:21.440573 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:21.440718 (+ 145us) server_negotiation.cc:407] Connection header received
0807 15:10:21.440835 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:21.440838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:21.440897 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:21.440977 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:21.441534 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:21.442136 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:21.443158 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:21.443424 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:21.443526 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:21.446559 (+ 3033us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.67:42163: BlockingRecv error: recv got EOF from 127.15.226.67:42163 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 07 15:10:22 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:22 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:22.022066 29220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:22.016221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.65:41783)
0807 15:10:22.016536 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:22.016555 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:22.016674 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:22.016934 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:22.016936 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:22.017144 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:22.017248 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:22.017253 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:22.018041 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:22.018044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:22.018605 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:22.018612 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:22.019152 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:22.019155 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:22.019219 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:22.019745 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:22.019762 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:22.021945 (+ 2183us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":88,"threads_started":1}
W20260807 15:10:22.022267 29223 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:22.016361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:41783 (local address 127.15.226.126:34719)
0807 15:10:22.016677 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:22.016680 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:22.016696 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:10:22.016740 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:22.016743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:22.016784 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:22.016868 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:22.017358 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:22.017935 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:22.018781 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:22.019045 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:22.019131 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:22.022089 (+ 2958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.65:41783: BlockingRecv error: recv got EOF from 127.15.226.65:41783 (error 108)
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":137,"threads_started":1}
Aug 07 15:10:22 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:22 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:22.232165 29224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:22.225753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.66:47267)
0807 15:10:22.226112 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:22.226128 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:22.226250 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:22.226585 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:22.226587 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:22.226832 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:22.226985 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:22.226993 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:22.228067 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:22.228071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:22.228673 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:22.228682 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:22.229124 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:22.229127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:22.229192 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:22.229846 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:22.229858 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:22.232034 (+ 2176us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":145,"threads_started":1}
W20260807 15:10:22.232375 29223 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:22.225864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:47267 (local address 127.15.226.126:34719)
0807 15:10:22.226024 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:22.226027 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:22.226206 (+ 179us) server_negotiation.cc:407] Connection header received
0807 15:10:22.226334 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:22.226336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:22.226441 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:22.226550 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:22.227127 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:22.227926 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:22.228784 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:22.229010 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:22.229102 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:22.232198 (+ 3096us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.66:47267: BlockingRecv error: recv got EOF from 127.15.226.66:47267 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 15:10:22 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:22 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:22.453270 29226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:22.447307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.67:46505)
0807 15:10:22.447678 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:22.447694 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:22.447901 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:22.448190 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:22.448193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:22.448440 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:22.448548 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:22.448554 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:22.449333 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:22.449338 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:22.449915 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:22.449923 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:22.450408 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:22.450412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:22.450502 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:22.451163 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:22.451183 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:22.453083 (+ 1900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":124,"threads_started":1}
W20260807 15:10:22.453351 29223 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:22.447449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:46505 (local address 127.15.226.126:34719)
0807 15:10:22.447630 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:22.447635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:22.447835 (+ 200us) server_negotiation.cc:407] Connection header received
0807 15:10:22.448010 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:22.448014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:22.448070 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:22.448161 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:22.448667 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:22.449211 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:22.450044 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:22.450285 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:22.450393 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:22.453223 (+ 2830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.67:46505: BlockingRecv error: recv got EOF from 127.15.226.67:46505 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 15:10:23 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:23 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:23.033663 29229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:23.023260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.65:42573)
0807 15:10:23.023653 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:23.023669 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:23.023784 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:23.024141 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:23.024144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:23.024348 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:23.024474 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:23.024480 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:23.025344 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:23.025346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:23.026092 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:23.026105 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:23.026740 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:23.026744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:23.026831 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:23.031220 (+ 4389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:23.031243 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:23.033501 (+ 2258us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":110,"threads_started":1}
W20260807 15:10:23.033898 29230 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:23.023350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:42573 (local address 127.15.226.126:34719)
0807 15:10:23.023628 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:23.023632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:23.023749 (+ 117us) server_negotiation.cc:407] Connection header received
0807 15:10:23.023945 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:23.023948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:23.023999 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:23.024096 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:23.024628 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:23.025229 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:23.026227 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:23.026498 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:23.026584 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:23.033702 (+ 7118us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.65:42573: BlockingRecv error: recv got EOF from 127.15.226.65:42573 (error 108)
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":106,"threads_started":1}
Aug 07 15:10:23 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:23 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:23.239617 29232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:23.233394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.66:39771)
0807 15:10:23.233797 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:23.233813 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:23.233923 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:23.234281 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:23.234285 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:23.234544 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:23.234748 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:23.234759 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:23.235737 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:23.235739 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:23.236310 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:23.236318 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:23.236839 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:23.236841 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:23.236915 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:23.237484 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:23.237497 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:23.239462 (+ 1965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":122,"threads_started":1}
W20260807 15:10:23.239781 29230 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:23.233484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:39771 (local address 127.15.226.126:34719)
0807 15:10:23.233697 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:23.233704 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:23.233904 (+ 200us) server_negotiation.cc:407] Connection header received
0807 15:10:23.234045 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:23.234048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:23.234095 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:23.234215 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:23.234869 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:23.235623 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:23.236482 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:23.236722 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:23.236818 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:23.239649 (+ 2831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.66:39771: BlockingRecv error: recv got EOF from 127.15.226.66:39771 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 07 15:10:23 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:23 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:23.460845 29233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:23.454416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.67:46425)
0807 15:10:23.454802 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:23.454867 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:23.455006 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:23.455295 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:23.455297 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:23.455533 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:23.455670 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:23.455678 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:23.456493 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:23.456497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:23.457078 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:23.457087 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:23.457618 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:23.457622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:23.457725 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:23.458388 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:23.458411 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:23.460626 (+ 2215us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":115,"threads_started":1}
W20260807 15:10:23.460947 29230 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:23.454453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:46425 (local address 127.15.226.126:34719)
0807 15:10:23.454627 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:23.454634 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:23.454978 (+ 344us) server_negotiation.cc:407] Connection header received
0807 15:10:23.455112 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:23.455115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:23.455174 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:23.455268 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:23.455807 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:23.456368 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:23.457209 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:23.457461 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:23.457574 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:23.460813 (+ 3239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.67:46425: BlockingRecv error: recv got EOF from 127.15.226.67:46425 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 07 15:10:24 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:24 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:24.046865 29235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:24.035161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.65:46537)
0807 15:10:24.035696 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:24.035725 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:24.035947 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:24.036434 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:24.036437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:24.036832 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:24.037013 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:24.037022 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:24.038454 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:24.038462 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:24.039784 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:24.039840 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:24.040832 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:24.040836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:24.040955 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:24.042329 (+ 1374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:24.042357 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:24.046537 (+ 4180us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":162,"threads_started":1}
W20260807 15:10:24.047220 29237 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:24.035377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:46537 (local address 127.15.226.126:34719)
0807 15:10:24.035762 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:24.035766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:24.035887 (+ 121us) server_negotiation.cc:407] Connection header received
0807 15:10:24.036101 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:24.036103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:24.036166 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:24.036310 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:24.037261 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:24.038177 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:24.040060 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:24.040618 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:24.040871 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:24.046953 (+ 6082us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.65:46537: BlockingRecv error: recv got EOF from 127.15.226.65:46537 (error 108)
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":155,"threads_started":1}
Aug 07 15:10:24 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:24 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:24.254189 29238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:24.241085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.66:59439)
0807 15:10:24.241589 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:24.241610 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:24.241792 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:24.242240 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:24.242246 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:24.242735 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:24.242954 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:24.242963 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:24.244251 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:24.244254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:24.245123 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:24.245133 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:24.245907 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:24.245919 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:24.246010 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:24.250226 (+ 4216us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:24.250254 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:24.253827 (+ 3573us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":143,"threads_started":1}
W20260807 15:10:24.254400 29237 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:24.241120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:59439 (local address 127.15.226.126:34719)
0807 15:10:24.241358 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:24.241367 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:24.241753 (+ 386us) server_negotiation.cc:407] Connection header received
0807 15:10:24.241935 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:24.241938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:24.242044 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:24.242162 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:24.243084 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:24.244067 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:24.245323 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:24.245712 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:24.245817 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:24.254146 (+ 8329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.66:59439: BlockingRecv error: recv got EOF from 127.15.226.66:59439 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 07 15:10:24 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:24 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 15:10:24.493672 29237 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:24.464307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:35611 (local address 127.15.226.126:34719)
0807 15:10:24.464575 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:24.464581 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:24.464614 (+ 33us) server_negotiation.cc:407] Connection header received
0807 15:10:24.464984 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:24.464994 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:24.465092 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:24.465551 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:24.468525 (+ 2974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:24.470374 (+ 1849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:24.476822 (+ 6448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:24.477498 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:24.477738 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:24.493348 (+ 15610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.67:35611: BlockingRecv error: recv got EOF from 127.15.226.67:35611 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 15:10:24.493810 29239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:24.463073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.67:35611)
0807 15:10:24.464284 (+ 1211us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:24.464321 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:24.464584 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:24.465677 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:24.465693 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:24.466829 (+ 1136us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:24.467665 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:24.467727 (+ 62us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:24.470732 (+ 3005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:24.470742 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:24.476043 (+ 5301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:24.476152 (+ 109us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:24.477757 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:24.477763 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:24.477942 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:24.479943 (+ 2001us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:24.480051 (+ 108us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:24.492827 (+ 12776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":262,"threads_started":1}
Aug 07 15:10:25 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:25 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:25.056257 29243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:25.048335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.65:48545)
0807 15:10:25.048721 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:25.048743 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:25.048942 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:25.049316 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:25.049319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:25.049581 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:25.049743 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:25.049749 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:25.050834 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:25.050837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:25.051580 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:25.051594 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:25.052204 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:25.052207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:25.052291 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:25.053018 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:25.053032 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:25.056077 (+ 3045us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":135,"threads_started":1}
W20260807 15:10:25.056614 29245 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:25.048485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:48545 (local address 127.15.226.126:34719)
0807 15:10:25.048799 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:25.048804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:25.048901 (+ 97us) server_negotiation.cc:407] Connection header received
0807 15:10:25.049059 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:25.049062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:25.049120 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:25.049222 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:25.049938 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:25.050584 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:25.051769 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:25.052065 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:25.052185 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:25.056424 (+ 4239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.65:48545: BlockingRecv error: recv got EOF from 127.15.226.65:48545 (error 108)
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":128,"threads_started":1}
Aug 07 15:10:25 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:25 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:25.262720 29246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:25.255620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.66:59659)
0807 15:10:25.255984 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:25.256005 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:25.256148 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:25.256487 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:25.256491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:25.256739 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:25.256863 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:25.256869 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:25.257736 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:25.257738 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:25.258368 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:25.258377 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:25.259045 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:25.259049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:25.259141 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:25.259774 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:25.259787 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:25.262427 (+ 2640us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":126,"threads_started":1}
W20260807 15:10:25.262786 29245 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:25.255784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:59659 (local address 127.15.226.126:34719)
0807 15:10:25.255977 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:25.255981 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:25.256112 (+ 131us) server_negotiation.cc:407] Connection header received
0807 15:10:25.256247 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:25.256250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:25.256316 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:25.256412 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:25.257043 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:25.257622 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:25.258538 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:25.258895 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:25.258997 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:25.262618 (+ 3621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.66:59659: BlockingRecv error: recv got EOF from 127.15.226.66:59659 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 07 15:10:25 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:25 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:25.502524 29250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:25.495809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.67:54939)
0807 15:10:25.496150 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:25.496166 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:25.496287 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:25.496566 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:25.496568 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:25.496772 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:25.496878 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:25.496884 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:25.497656 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:25.497658 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:25.498245 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:25.498255 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:25.498865 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:25.498870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:25.498942 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:25.499652 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:25.499665 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:25.502373 (+ 2708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":114,"threads_started":1}
W20260807 15:10:25.502522 29245 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:25.495927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:54939 (local address 127.15.226.126:34719)
0807 15:10:25.496132 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:25.496136 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:25.496251 (+ 115us) server_negotiation.cc:407] Connection header received
0807 15:10:25.496375 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:25.496378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:25.496444 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:25.496523 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:25.496998 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:25.497548 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:25.498371 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:25.498700 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:25.498838 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:25.502408 (+ 3570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.67:54939: BlockingRecv error: recv got EOF from 127.15.226.67:54939 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 07 15:10:26 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:26 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:26.064378 29251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:26.057430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.65:54219)
0807 15:10:26.057771 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:26.057789 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:26.057891 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:26.058309 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:26.058311 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:26.058535 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:26.058642 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:26.058688 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:26.059710 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:26.059714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:26.060336 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:26.060348 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:26.060893 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:26.060897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:26.060973 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:26.061549 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:26.061562 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:26.064171 (+ 2609us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":135,"threads_started":1}
W20260807 15:10:26.064631 29252 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:26.057568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:54219 (local address 127.15.226.126:34719)
0807 15:10:26.057855 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:26.057858 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:26.057876 (+ 18us) server_negotiation.cc:407] Connection header received
0807 15:10:26.058109 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:26.058112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:26.058166 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:26.058267 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:26.058858 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:26.059549 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:26.060494 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:26.060743 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:26.060835 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:26.064350 (+ 3515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.65:54219: BlockingRecv error: recv got EOF from 127.15.226.65:54219 (error 108)
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":109,"threads_started":1}
Aug 07 15:10:26 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:26 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:26.270016 29253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:26.263665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.66:56433)
0807 15:10:26.263976 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:26.263991 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:26.264102 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:26.264357 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:26.264359 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:26.264670 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:26.264845 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:26.264853 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:26.265766 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:26.265769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:26.266412 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:26.266422 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:26.267009 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:26.267012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:26.267131 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:26.267722 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:26.267737 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:26.269835 (+ 2098us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":120,"threads_started":1}
W20260807 15:10:26.270143 29252 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:26.263771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:56433 (local address 127.15.226.126:34719)
0807 15:10:26.263921 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:26.263925 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:26.264047 (+ 122us) server_negotiation.cc:407] Connection header received
0807 15:10:26.264187 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:26.264190 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:26.264240 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:26.264306 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:26.265010 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:26.265563 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:26.266539 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:26.266878 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:26.266965 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:26.270024 (+ 3059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.66:56433: BlockingRecv error: recv got EOF from 127.15.226.66:56433 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 07 15:10:26 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:26 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:26.510337 29254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:26.503648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.67:50073)
0807 15:10:26.504006 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:26.504027 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:26.504125 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:26.504539 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:26.504567 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:26.504877 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:26.505026 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:26.505035 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:26.506183 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:26.506187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:26.506874 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:26.506885 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:26.507390 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:26.507393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:26.507466 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:26.508069 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:26.508083 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:26.510212 (+ 2129us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":151,"threads_started":1}
W20260807 15:10:26.510563 29252 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:26.503924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:50073 (local address 127.15.226.126:34719)
0807 15:10:26.504077 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:26.504080 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:26.504099 (+ 19us) server_negotiation.cc:407] Connection header received
0807 15:10:26.504316 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:26.504321 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:26.504386 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:26.504498 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:26.505188 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:26.506037 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:26.507045 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:26.507275 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:26.507370 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:26.510410 (+ 3040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.67:50073: BlockingRecv error: recv got EOF from 127.15.226.67:50073 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 15:10:27 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:27 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:27.072332 29255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:27.065413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.65:33463)
0807 15:10:27.065776 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:27.065793 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:27.065911 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:27.066178 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:27.066180 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:27.066406 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:27.066518 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:27.066525 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:27.067483 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:27.067486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:27.068157 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:27.068169 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:27.068671 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:27.068673 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:27.068747 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:27.069475 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:27.069488 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:27.072119 (+ 2631us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":156,"threads_started":1}
W20260807 15:10:27.072373 29257 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:27.065538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:33463 (local address 127.15.226.126:34719)
0807 15:10:27.065811 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:27.065815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:27.065857 (+ 42us) server_negotiation.cc:407] Connection header received
0807 15:10:27.065992 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:27.065994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:27.066039 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:27.066133 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:27.066704 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:27.067346 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:27.068301 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:27.068553 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:27.068638 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:27.072250 (+ 3612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.65:33463: BlockingRecv error: recv got EOF from 127.15.226.65:33463 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":97,"threads_started":1}
Aug 07 15:10:27 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:27 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:27.277747 29260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:27.271089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.66:53491)
0807 15:10:27.271437 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:27.271453 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:27.271609 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:27.271987 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:27.271991 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:27.272260 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:27.272385 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:27.272391 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:27.273391 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:27.273396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:27.274037 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:27.274047 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:27.274599 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:27.274603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:27.274758 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:27.275445 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:27.275459 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:27.277617 (+ 2158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":108,"threads_started":1}
W20260807 15:10:27.278038 29257 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:27.271196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:53491 (local address 127.15.226.126:34719)
0807 15:10:27.271381 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:27.271386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:27.271536 (+ 150us) server_negotiation.cc:407] Connection header received
0807 15:10:27.271771 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:27.271778 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:27.271847 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:27.271962 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:27.272520 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:27.273260 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:27.274184 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:27.274450 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:27.274574 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:27.277872 (+ 3298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.66:53491: BlockingRecv error: recv got EOF from 127.15.226.66:53491 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 15:10:27 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:27 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:27.517309 29262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:27.511227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.67:34559)
0807 15:10:27.511625 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:27.511639 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:27.511803 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:27.512099 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:27.512102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:27.512325 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:27.512424 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:27.512430 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:27.513256 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:27.513260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:27.513838 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:27.513846 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:27.514373 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:27.514377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:27.514468 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:27.515123 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:27.515136 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:27.517109 (+ 1973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":190,"threads_started":1}
W20260807 15:10:27.517405 29257 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:27.511245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:34559 (local address 127.15.226.126:34719)
0807 15:10:27.511425 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:27.511431 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:27.511738 (+ 307us) server_negotiation.cc:407] Connection header received
0807 15:10:27.511911 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:27.511916 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:27.511977 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:27.512070 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:27.512550 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:27.513133 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:27.514002 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:27.514242 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:27.514356 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:27.517274 (+ 2918us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.67:34559: BlockingRecv error: recv got EOF from 127.15.226.67:34559 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 15:10:28 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:28 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:28.084470 29264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:28.073452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.65:40077)
0807 15:10:28.073797 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:28.073816 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:28.073941 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:28.074350 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:28.074353 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:28.074562 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:28.074866 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:28.074881 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:28.075696 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:28.075699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:28.076274 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:28.076282 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:28.076720 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:28.076723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:28.076787 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:28.081663 (+ 4876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:28.081677 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:28.084276 (+ 2599us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":115,"threads_started":1}
W20260807 15:10:28.084560 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:28.073523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:40077 (local address 127.15.226.126:34719)
0807 15:10:28.073810 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:28.073816 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:28.073887 (+ 71us) server_negotiation.cc:407] Connection header received
0807 15:10:28.074133 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:28.074136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:28.074187 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:28.074278 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:28.075016 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:28.075571 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:28.076383 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:28.076612 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:28.076691 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:28.084427 (+ 7736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.65:40077: BlockingRecv error: recv got EOF from 127.15.226.65:40077 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":74,"threads_started":1}
Aug 07 15:10:28 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:28 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:28.284088 29267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:28.278570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.66:46219)
0807 15:10:28.278884 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:28.278900 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:28.279026 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:28.279272 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:28.279274 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:28.279483 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:28.279589 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:28.279595 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:28.280372 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:28.280374 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:28.280946 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:28.280955 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:28.281432 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:28.281436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:28.281502 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:28.282031 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:28.282043 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:28.283962 (+ 1919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":133,"threads_started":1}
W20260807 15:10:28.284260 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:28.278743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:46219 (local address 127.15.226.126:34719)
0807 15:10:28.278902 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:28.278905 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:28.278990 (+ 85us) server_negotiation.cc:407] Connection header received
0807 15:10:28.279108 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:28.279110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:28.279154 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:28.279232 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:28.279694 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:28.280239 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:28.281070 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:28.281316 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:28.281399 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:28.284120 (+ 2721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.66:46219: BlockingRecv error: recv got EOF from 127.15.226.66:46219 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 07 15:10:28 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:28 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:28.523952 29268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:28.518354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.67:38925)
0807 15:10:28.518761 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:28.518775 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:28.518888 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:28.519118 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:28.519119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:28.519314 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:28.519413 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:28.519417 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:28.520177 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:28.520180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:28.520749 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:28.520757 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:28.521210 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:28.521213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:28.521286 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:28.521859 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:28.521871 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:28.523831 (+ 1960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":113,"threads_started":1}
W20260807 15:10:28.524122 29265 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:28.518385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:38925 (local address 127.15.226.126:34719)
0807 15:10:28.518539 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:28.518545 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:28.518856 (+ 311us) server_negotiation.cc:407] Connection header received
0807 15:10:28.518981 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:28.518984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:28.519035 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:28.519106 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:28.519514 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:28.520050 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:28.520870 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:28.521095 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:28.521172 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:28.523986 (+ 2814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.67:38925: BlockingRecv error: recv got EOF from 127.15.226.67:38925 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 07 15:10:29 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:29 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:29.091732 29269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:29.085439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.65:42049)
0807 15:10:29.085836 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:29.085854 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:29.085976 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:29.086240 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:29.086242 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:29.086454 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:29.086564 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:29.086569 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:29.087443 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:29.087446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:29.088093 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:29.088104 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:29.088598 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:29.088601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:29.088674 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:29.089288 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:29.089301 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:29.091527 (+ 2226us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":109,"threads_started":1}
W20260807 15:10:29.091852 29270 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:29.085619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:42049 (local address 127.15.226.126:34719)
0807 15:10:29.085878 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:29.085882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:29.085958 (+ 76us) server_negotiation.cc:407] Connection header received
0807 15:10:29.086063 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:29.086066 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:29.086112 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:29.086198 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:29.086730 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:29.087309 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:29.088218 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:29.088460 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:29.088546 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:29.091698 (+ 3152us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.65:42049: BlockingRecv error: recv got EOF from 127.15.226.65:42049 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":78,"threads_started":1}
Aug 07 15:10:29 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:29 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:29.291517 29271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:29.285131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.66:48409)
0807 15:10:29.285455 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:29.285472 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:29.285580 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:29.285863 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:29.285865 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:29.286071 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:29.286176 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:29.286182 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:29.287007 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:29.287010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:29.287700 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:29.287775 (+ 75us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:29.288415 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:29.288419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:29.288490 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:29.289233 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:29.289246 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:29.291380 (+ 2134us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":124,"threads_started":1}
W20260807 15:10:29.291657 29270 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:29.285375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:48409 (local address 127.15.226.126:34719)
0807 15:10:29.285533 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:29.285537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:29.285553 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:10:29.285678 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:29.285680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:29.285728 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:29.285825 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:29.286297 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:29.286867 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:29.287923 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:29.288240 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:29.288356 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:29.291538 (+ 3182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.66:48409: BlockingRecv error: recv got EOF from 127.15.226.66:48409 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 15:10:29 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:29 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:29.531652 29272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:29.525003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.67:48633)
0807 15:10:29.525337 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:29.525359 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:29.525462 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:29.525770 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:29.525772 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:29.525974 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:29.526077 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:29.526083 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:29.527011 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:29.527015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:29.527730 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:29.527742 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:29.528426 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:29.528429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:29.528500 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:29.529137 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:29.529150 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:29.531482 (+ 2332us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":147,"threads_started":1}
W20260807 15:10:29.531857 29270 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:29.525176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:48633 (local address 127.15.226.126:34719)
0807 15:10:29.525486 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:29.525490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:29.525508 (+ 18us) server_negotiation.cc:407] Connection header received
0807 15:10:29.525555 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:29.525557 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:29.525604 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:29.525697 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:29.526203 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:29.526889 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:29.527878 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:29.528139 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:29.528250 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:29.531680 (+ 3430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.67:48633: BlockingRecv error: recv got EOF from 127.15.226.67:48633 (error 108)
Metrics: {"server-negotiator.queue_time_us":210}
Aug 07 15:10:30 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:30 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:30.100821 29274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:30.092810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.65:57785)
0807 15:10:30.093133 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:30.093149 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:30.093260 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:30.093572 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:30.093575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:30.093847 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:30.093956 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:30.093963 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:30.094937 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:30.094943 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:30.096093 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:30.096112 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:30.096876 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:30.096883 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:30.097017 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:30.097830 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:30.097854 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:30.100649 (+ 2795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":126,"threads_started":1}
W20260807 15:10:30.101148 29275 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:30.092966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:57785 (local address 127.15.226.126:34719)
0807 15:10:30.093337 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:30.093341 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:30.093355 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:10:30.093397 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:30.093399 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:30.093442 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:30.093533 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:30.094118 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:30.094767 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:30.096280 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:30.096699 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:30.096851 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:30.100968 (+ 4117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.65:57785: BlockingRecv error: recv got EOF from 127.15.226.65:57785 (error 108)
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":112,"threads_started":1}
Aug 07 15:10:30 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:30 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:30.299232 29278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:30.292686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.66:36783)
0807 15:10:30.293039 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:30.293055 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:30.293178 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:30.293438 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:30.293440 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:30.293647 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:30.293749 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:30.293754 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:30.294570 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:30.294573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:30.295228 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:30.295239 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:30.295824 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:30.295827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:30.295900 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:30.296595 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:30.296608 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:30.299034 (+ 2426us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":95,"threads_started":1}
W20260807 15:10:30.299294 29275 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:30.292805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:36783 (local address 127.15.226.126:34719)
0807 15:10:30.293030 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:30.293034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:30.293142 (+ 108us) server_negotiation.cc:407] Connection header received
0807 15:10:30.293266 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:30.293269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:30.293315 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:30.293394 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:30.293900 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:30.294449 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:30.295381 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:30.295628 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:30.295794 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:30.299174 (+ 3380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.66:36783: BlockingRecv error: recv got EOF from 127.15.226.66:36783 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
Aug 07 15:10:30 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:30 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.67@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:30.539283 29279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:30.532664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.67:34281)
0807 15:10:30.532976 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:30.532991 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:30.533110 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:30.533456 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:30.533463 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:30.533683 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:30.533849 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:30.533857 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:30.534870 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:30.534874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:30.535487 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:30.535497 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:30.536032 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:30.536036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:30.536125 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:30.536838 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:30.536852 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:30.539109 (+ 2257us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":95,"threads_started":1}
W20260807 15:10:30.539518 29275 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:30.532754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:34281 (local address 127.15.226.126:34719)
0807 15:10:30.532902 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:30.532906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:30.533067 (+ 161us) server_negotiation.cc:407] Connection header received
0807 15:10:30.533188 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:30.533190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:30.533240 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:30.533324 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:30.534059 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:30.534717 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:30.535636 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:30.535894 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:30.536019 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:30.539367 (+ 3348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.67:34281: BlockingRecv error: recv got EOF from 127.15.226.67:34281 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 07 15:10:31 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:31 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.65@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:31.108042 29280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:31.102023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.65:51991)
0807 15:10:31.102357 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:31.102372 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:31.102473 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:31.102828 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:31.102830 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:31.103038 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:31.103142 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:31.103148 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:31.104071 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:31.104074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:31.104641 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:31.104650 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:31.105119 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:31.105121 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:31.105187 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:31.105804 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:31.105817 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:31.107917 (+ 2100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":93,"threads_started":1}
W20260807 15:10:31.108392 29282 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:31.102118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:51991 (local address 127.15.226.126:34719)
0807 15:10:31.102392 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:31.102396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:31.102430 (+ 34us) server_negotiation.cc:407] Connection header received
0807 15:10:31.102560 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:31.102563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:31.102614 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:31.102820 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:31.103308 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:31.103957 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:31.104766 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:31.105028 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:31.105110 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:31.108098 (+ 2988us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.65:51991: BlockingRecv error: recv got EOF from 127.15.226.65:51991 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":95,"threads_started":1}
Aug 07 15:10:31 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 15:10:31 dist-test-slave-3kk6 krb5kdc[27798](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.226.66@KRBTEST.COM for oryx/127.15.226.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 15:10:31.306311 29283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:31.300328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34719 (local address 127.15.226.66:47813)
0807 15:10:31.300615 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:31.300629 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:31.300730 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:31.300973 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:31.300975 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:31.301170 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:31.301281 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:31.301286 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:31.302098 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:31.302104 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:31.302965 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:31.302976 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:31.303419 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:31.303422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:31.303484 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:31.304152 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:31.304165 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:31.306162 (+ 1997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.226.126:34719: Server oryx/127.15.226.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":104,"threads_started":1}
W20260807 15:10:31.306519 29282 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:31.300403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:47813 (local address 127.15.226.126:34719)
0807 15:10:31.300547 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:31.300551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:31.300705 (+ 154us) server_negotiation.cc:407] Connection header received
0807 15:10:31.300809 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:31.300811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:31.300855 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:31.300932 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:31.301388 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:31.301968 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:31.303080 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:31.303315 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:31.303399 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:31.306346 (+ 2947us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.226.66:47813: BlockingRecv error: recv got EOF from 127.15.226.66:47813 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 15:10:31.404837 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 28623
I20260807 15:10:31.420559 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 28821
I20260807 15:10:31.433635 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 29004
I20260807 15:10:31.447686 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 28529
2026-08-07T15:10:31Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18509 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T15:10:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:31Z Disabled control of system clock
I20260807 15:10:31.495034 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:38649
--webserver_interface=127.15.226.126
--webserver_port=0
--builtin_ntp_servers=127.15.226.84:38393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:38649
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 15:10:31.631330 29288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:31.631650 29288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:31.631703 29288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:31.635957 29288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:31.636047 29288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:31.636063 29288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:31.636081 29288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:31.636097 29288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:31.641165 29288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:38649
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:38649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:31.642591 29288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:31.644169 29288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:31.654771 29298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:31.654786 29296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:31.654851 29288 server_base.cc:1034] running on GCE node
W20260807 15:10:31.654853 29300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:31.655709 29288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:31.657039 29288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:31.658260 29288 hybrid_clock.cc:648] HybridClock initialized: now 1786115431658234 us; error 49 us; skew 500 ppm
I20260807 15:10:31.660591 29288 webserver.cc:460] Webserver started at http://127.15.226.126:42219/ using document root <none> and password file <none>
I20260807 15:10:31.661325 29288 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:31.661432 29288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:31.661700 29288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:31.664110 29288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "a3f09262a8ff4cf8aaaad47050362af1"
format_stamp: "Formatted at 2026-08-07 15:10:31 on dist-test-slave-3kk6"
server_key: "bae1544764a9c29bf8cd3734815c5e4f"
server_key_iv: "5f6ad512a47e9ba8e43e67ab5e8a85ce"
server_key_version: "encryptionkey@0"
I20260807 15:10:31.664788 29288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "a3f09262a8ff4cf8aaaad47050362af1"
format_stamp: "Formatted at 2026-08-07 15:10:31 on dist-test-slave-3kk6"
server_key: "bae1544764a9c29bf8cd3734815c5e4f"
server_key_iv: "5f6ad512a47e9ba8e43e67ab5e8a85ce"
server_key_version: "encryptionkey@0"
I20260807 15:10:31.669399 29288 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:10:31.672519 29310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:31.674206 29288 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:10:31.674401 29288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "a3f09262a8ff4cf8aaaad47050362af1"
format_stamp: "Formatted at 2026-08-07 15:10:31 on dist-test-slave-3kk6"
server_key: "bae1544764a9c29bf8cd3734815c5e4f"
server_key_iv: "5f6ad512a47e9ba8e43e67ab5e8a85ce"
server_key_version: "encryptionkey@0"
I20260807 15:10:31.674561 29288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:31.692648 29288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:31.693558 29288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:31.693819 29288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:31.702700 29288 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:38649
I20260807 15:10:31.702780 29384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:38649 every 8 connection(s)
I20260807 15:10:31.703886 29288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:31.708317 29385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:31.713685 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 29288
I20260807 15:10:31.713853 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:10:31.714239 16265 external_mini_cluster.cc:1428] Setting key 90cb7e6d4e83e8b1d2e71d1eab767465
I20260807 15:10:31.716404 29385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3f09262a8ff4cf8aaaad47050362af1: Bootstrap starting.
I20260807 15:10:31.719276 29385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3f09262a8ff4cf8aaaad47050362af1: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:31.720172 29385 log.cc:824] T 00000000000000000000000000000000 P a3f09262a8ff4cf8aaaad47050362af1: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:31.722345 29385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3f09262a8ff4cf8aaaad47050362af1: No bootstrap required, opened a new log
I20260807 15:10:31.724746 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:31.716013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53832 (local address 127.15.226.126:38649)
0807 15:10:31.716893 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:31.716913 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:31.716956 (+ 43us) server_negotiation.cc:407] Connection header received
0807 15:10:31.717751 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:31.717770 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:31.718200 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:31.718575 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:31.719096 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:31.720175 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:31.721180 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:31.721676 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:31.721872 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:31.722848 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:31.722866 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:31.722879 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:31.723299 (+ 420us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:31.723358 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:31.723392 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:31.723481 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:31.723677 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 15:10:31.724010 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":182,"threads_started":1}
I20260807 15:10:31.726634 29385 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3f09262a8ff4cf8aaaad47050362af1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3f09262a8ff4cf8aaaad47050362af1" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 38649 } }
I20260807 15:10:31.726969 29385 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3f09262a8ff4cf8aaaad47050362af1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:31.727056 29385 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3f09262a8ff4cf8aaaad47050362af1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3f09262a8ff4cf8aaaad47050362af1, State: Initialized, Role: FOLLOWER
I20260807 15:10:31.727617 29385 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3f09262a8ff4cf8aaaad47050362af1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3f09262a8ff4cf8aaaad47050362af1" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 38649 } }
I20260807 15:10:31.727794 29385 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3f09262a8ff4cf8aaaad47050362af1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:31.727881 29385 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3f09262a8ff4cf8aaaad47050362af1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:31.728016 29385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3f09262a8ff4cf8aaaad47050362af1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:31.729161 29385 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3f09262a8ff4cf8aaaad47050362af1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3f09262a8ff4cf8aaaad47050362af1" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 38649 } }
I20260807 15:10:31.729592 29385 leader_election.cc:302] T 00000000000000000000000000000000 P a3f09262a8ff4cf8aaaad47050362af1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3f09262a8ff4cf8aaaad47050362af1; no voters:
I20260807 15:10:31.730021 29385 leader_election.cc:288] T 00000000000000000000000000000000 P a3f09262a8ff4cf8aaaad47050362af1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:10:31.730105 29391 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3f09262a8ff4cf8aaaad47050362af1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:31.730325 29391 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3f09262a8ff4cf8aaaad47050362af1 [term 1 LEADER]: Becoming Leader. State: Replica: a3f09262a8ff4cf8aaaad47050362af1, State: Running, Role: LEADER
I20260807 15:10:31.730762 29391 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3f09262a8ff4cf8aaaad47050362af1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3f09262a8ff4cf8aaaad47050362af1" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 38649 } }
I20260807 15:10:31.731462 29385 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3f09262a8ff4cf8aaaad47050362af1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:31.732846 29393 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3f09262a8ff4cf8aaaad47050362af1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3f09262a8ff4cf8aaaad47050362af1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3f09262a8ff4cf8aaaad47050362af1" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 38649 } } }
I20260807 15:10:31.733043 29393 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3f09262a8ff4cf8aaaad47050362af1 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:31.732874 29392 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3f09262a8ff4cf8aaaad47050362af1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3f09262a8ff4cf8aaaad47050362af1. Latest consensus state: current_term: 1 leader_uuid: "a3f09262a8ff4cf8aaaad47050362af1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3f09262a8ff4cf8aaaad47050362af1" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 38649 } } }
I20260807 15:10:31.733211 29392 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3f09262a8ff4cf8aaaad47050362af1 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:31.734444 29408 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:31.737866 29408 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:10:31.744984 29408 catalog_manager.cc:1347] Generated new cluster ID: f3c4223bea414895843984052830e8ec
I20260807 15:10:31.745082 29408 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:10:31.782881 29408 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:10:31.784276 29408 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:31.790438 29408 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3f09262a8ff4cf8aaaad47050362af1: Generated new TSK 0
I20260807 15:10:31.791332 29408 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:10:31.795500 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:38649
--builtin_ntp_servers=127.15.226.84:38393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 15:10:31.923462 29422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:31.923827 29422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:31.923914 29422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:31.928153 29422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:31.928263 29422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:31.928369 29422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:10:31.933684 29422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:38649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:31.934952 29422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:31.936498 29422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:31.947592 29430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:31.947569 29429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:31.947569 29434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:31.947930 29422 server_base.cc:1034] running on GCE node
I20260807 15:10:31.948596 29422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:31.949335 29422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:31.950553 29422 hybrid_clock.cc:648] HybridClock initialized: now 1786115431950531 us; error 37 us; skew 500 ppm
I20260807 15:10:31.952965 29422 webserver.cc:460] Webserver started at http://127.15.226.65:44255/ using document root <none> and password file <none>
I20260807 15:10:31.953640 29422 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:31.953749 29422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:31.954015 29422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:31.956548 29422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "fa8c1def6f7d4ec18dc464f9a404efef"
format_stamp: "Formatted at 2026-08-07 15:10:31 on dist-test-slave-3kk6"
server_key: "268feee87888cec900f03da035238f00"
server_key_iv: "4eb4174767027ed7e0dd52b83d5203a3"
server_key_version: "encryptionkey@0"
I20260807 15:10:31.957211 29422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "fa8c1def6f7d4ec18dc464f9a404efef"
format_stamp: "Formatted at 2026-08-07 15:10:31 on dist-test-slave-3kk6"
server_key: "268feee87888cec900f03da035238f00"
server_key_iv: "4eb4174767027ed7e0dd52b83d5203a3"
server_key_version: "encryptionkey@0"
I20260807 15:10:31.961495 29422 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 15:10:31.964481 29443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:31.965871 29422 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:10:31.966032 29422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "fa8c1def6f7d4ec18dc464f9a404efef"
format_stamp: "Formatted at 2026-08-07 15:10:31 on dist-test-slave-3kk6"
server_key: "268feee87888cec900f03da035238f00"
server_key_iv: "4eb4174767027ed7e0dd52b83d5203a3"
server_key_version: "encryptionkey@0"
I20260807 15:10:31.966176 29422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:31.991954 29422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:31.992893 29422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:31.993147 29422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:31.993892 29422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:31.995092 29422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:31.995177 29422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:31.995260 29422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:31.995317 29422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:32.006870 29422 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:43453
I20260807 15:10:32.006919 29616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:43453 every 8 connection(s)
I20260807 15:10:32.007956 29422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:10:32.014629 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 29422
I20260807 15:10:32.014791 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:10:32.015133 16265 external_mini_cluster.cc:1428] Setting key 0ca5c4c252a2e4e32ada178a1f09a52a
I20260807 15:10:32.017452 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:38649
--builtin_ntp_servers=127.15.226.84:38393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 15:10:32.018976 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:32.010086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:37971 (local address 127.15.226.126:38649)
0807 15:10:32.010243 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:32.010247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:32.011544 (+ 1297us) server_negotiation.cc:407] Connection header received
0807 15:10:32.013118 (+ 1574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:32.013121 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:32.013186 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:32.013272 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:32.014281 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:32.015174 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:32.016630 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:32.016922 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:32.017045 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:32.018314 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:32.018321 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:32.018327 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:32.018348 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:32.018378 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:32.018392 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:32.018484 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:32.018643 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 15:10:32.018748 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 15:10:32.019896 29621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:32.010583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:38649 (local address 127.15.226.65:37971)
0807 15:10:32.011362 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:32.011409 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:32.012860 (+ 1451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:32.013467 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:32.013474 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:32.013565 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:32.014114 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:32.014128 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:32.015340 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:32.015343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:32.016468 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:32.016482 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:32.017096 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:32.017099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:32.017261 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:32.018011 (+ 750us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:10:32.018012 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:32.018060 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:10:32.018135 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:32.018493 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:32.018500 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:32.018505 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 15:10:32.018610 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 15:10:32.018957 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":153,"threads_started":1}
I20260807 15:10:32.021292 29618 heartbeater.cc:347] Connected to a master server at 127.15.226.126:38649
I20260807 15:10:32.021570 29618 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:32.022162 29618 heartbeater.cc:511] Master 127.15.226.126:38649 requested a full tablet report, sending...
I20260807 15:10:32.023907 29335 ts_manager.cc:194] Registered new tserver with Master: fa8c1def6f7d4ec18dc464f9a404efef (127.15.226.65:43453)
I20260807 15:10:32.025175 29335 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.226.65:37971
W20260807 15:10:32.147442 29622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:32.147780 29622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:32.147823 29622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:32.152240 29622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:32.152338 29622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:32.152400 29622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:10:32.157845 29622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:38649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:32.159242 29622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:32.160884 29622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:10:32.171478 29622 server_base.cc:1034] running on GCE node
W20260807 15:10:32.171470 29630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:32.171746 29628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:32.171470 29632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:32.172340 29622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:32.173138 29622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:32.174367 29622 hybrid_clock.cc:648] HybridClock initialized: now 1786115432174342 us; error 45 us; skew 500 ppm
I20260807 15:10:32.176748 29622 webserver.cc:460] Webserver started at http://127.15.226.66:38213/ using document root <none> and password file <none>
I20260807 15:10:32.177470 29622 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:32.177575 29622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:32.177847 29622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:32.180640 29622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "8671788be0af437fbe728e79b6fdcd56"
format_stamp: "Formatted at 2026-08-07 15:10:32 on dist-test-slave-3kk6"
server_key: "6febf833b179bae3e4de9037607237da"
server_key_iv: "98539b8fde45d10ab28c2229a501cabc"
server_key_version: "encryptionkey@0"
I20260807 15:10:32.181344 29622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "8671788be0af437fbe728e79b6fdcd56"
format_stamp: "Formatted at 2026-08-07 15:10:32 on dist-test-slave-3kk6"
server_key: "6febf833b179bae3e4de9037607237da"
server_key_iv: "98539b8fde45d10ab28c2229a501cabc"
server_key_version: "encryptionkey@0"
I20260807 15:10:32.185575 29622 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260807 15:10:32.188621 29641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:32.190099 29622 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:10:32.190261 29622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "8671788be0af437fbe728e79b6fdcd56"
format_stamp: "Formatted at 2026-08-07 15:10:32 on dist-test-slave-3kk6"
server_key: "6febf833b179bae3e4de9037607237da"
server_key_iv: "98539b8fde45d10ab28c2229a501cabc"
server_key_version: "encryptionkey@0"
I20260807 15:10:32.190409 29622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:32.207927 29622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:32.208799 29622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:32.209054 29622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:32.209753 29622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:32.210891 29622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:32.210997 29622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:32.211071 29622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:32.211088 29622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:32.221453 29622 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:37371
I20260807 15:10:32.221495 29807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:37371 every 8 connection(s)
I20260807 15:10:32.222508 29622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:10:32.226784 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 29622
I20260807 15:10:32.226914 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:10:32.227314 16265 external_mini_cluster.cc:1428] Setting key 45c1d2199b5390c9cef4ba1d4a581df0
I20260807 15:10:32.229766 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:38649
--builtin_ntp_servers=127.15.226.84:38393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 15:10:32.233546 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:32.224655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:40621 (local address 127.15.226.126:38649)
0807 15:10:32.224796 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:32.224800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:32.225934 (+ 1134us) server_negotiation.cc:407] Connection header received
0807 15:10:32.227916 (+ 1982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:32.227920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:32.227986 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:32.228108 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:32.229135 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:32.229678 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:32.231090 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:32.231362 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:32.231492 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:32.232952 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:32.232957 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:32.232959 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:32.232975 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:32.232998 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:32.233014 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:32.233078 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:32.233299 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 15:10:32.233353 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 15:10:32.234622 29810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:32.225083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:38649 (local address 127.15.226.66:40621)
0807 15:10:32.225775 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:32.225818 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:32.227617 (+ 1799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:32.228320 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:32.228327 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:32.228423 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:32.228977 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:32.228990 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:32.229818 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:32.229821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:32.230958 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:32.230970 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:32.231543 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:32.231548 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:32.231740 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:32.232518 (+ 778us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:10:32.232522 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:32.232590 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:10:32.232739 (+ 149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:32.233115 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:32.233126 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:32.233132 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:10:32.233291 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 15:10:32.233755 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":135,"threads_started":1}
I20260807 15:10:32.236129 29808 heartbeater.cc:347] Connected to a master server at 127.15.226.126:38649
I20260807 15:10:32.236450 29808 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:32.237213 29808 heartbeater.cc:511] Master 127.15.226.126:38649 requested a full tablet report, sending...
I20260807 15:10:32.238632 29335 ts_manager.cc:194] Registered new tserver with Master: 8671788be0af437fbe728e79b6fdcd56 (127.15.226.66:37371)
I20260807 15:10:32.239420 29335 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.226.66:40621
W20260807 15:10:32.357785 29811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:32.358181 29811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:32.358261 29811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:32.362555 29811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:32.362720 29811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:32.362821 29811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:10:32.367976 29811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:38649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:32.369174 29811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:32.370783 29811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:32.381973 29818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:32.381976 29817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:32.382028 29821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:32.382313 29811 server_base.cc:1034] running on GCE node
I20260807 15:10:32.382970 29811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:32.383762 29811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:32.384986 29811 hybrid_clock.cc:648] HybridClock initialized: now 1786115432384967 us; error 35 us; skew 500 ppm
I20260807 15:10:32.387262 29811 webserver.cc:460] Webserver started at http://127.15.226.67:35291/ using document root <none> and password file <none>
I20260807 15:10:32.387924 29811 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:32.388026 29811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:32.388283 29811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:32.390760 29811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "43256b73ddf6405b9c63104c2ad4a4cd"
format_stamp: "Formatted at 2026-08-07 15:10:32 on dist-test-slave-3kk6"
server_key: "a0ffcc23f691ceb9b0640b9168fb45bc"
server_key_iv: "4583e371be422b39425014653eccc9cb"
server_key_version: "encryptionkey@0"
I20260807 15:10:32.391435 29811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "43256b73ddf6405b9c63104c2ad4a4cd"
format_stamp: "Formatted at 2026-08-07 15:10:32 on dist-test-slave-3kk6"
server_key: "a0ffcc23f691ceb9b0640b9168fb45bc"
server_key_iv: "4583e371be422b39425014653eccc9cb"
server_key_version: "encryptionkey@0"
I20260807 15:10:32.395872 29811 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:10:32.398838 29829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:32.400444 29811 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 15:10:32.400645 29811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "43256b73ddf6405b9c63104c2ad4a4cd"
format_stamp: "Formatted at 2026-08-07 15:10:32 on dist-test-slave-3kk6"
server_key: "a0ffcc23f691ceb9b0640b9168fb45bc"
server_key_iv: "4583e371be422b39425014653eccc9cb"
server_key_version: "encryptionkey@0"
I20260807 15:10:32.400817 29811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:32.416533 29811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:32.417367 29811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:32.417603 29811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:32.418296 29811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:32.419487 29811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:32.419561 29811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:32.419638 29811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:32.419687 29811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:32.429750 29811 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:35115
I20260807 15:10:32.429772 29995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:35115 every 8 connection(s)
I20260807 15:10:32.430886 29811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:10:32.440055 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 29811
I20260807 15:10:32.440220 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:10:32.440621 16265 external_mini_cluster.cc:1428] Setting key 8ad5e609dcbbe4939a4e21bb42d16f96
I20260807 15:10:32.441946 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:32.433012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:59345 (local address 127.15.226.126:38649)
0807 15:10:32.433179 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:32.433184 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:32.434517 (+ 1333us) server_negotiation.cc:407] Connection header received
0807 15:10:32.436051 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:32.436054 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:32.436108 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:32.436193 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:32.437313 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:32.437853 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:32.439381 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:32.439609 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:32.439706 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:32.441300 (+ 1594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:32.441305 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:32.441309 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:32.441326 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:32.441356 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:32.441365 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:32.441435 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:32.441687 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 15:10:32.441772 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 15:10:32.443250 29998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:32.433568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:38649 (local address 127.15.226.67:59345)
0807 15:10:32.434332 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:32.434378 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:32.435823 (+ 1445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:32.436444 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:32.436452 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:32.436551 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:32.437141 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:32.437157 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:32.437996 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:32.438000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:32.439248 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:32.439259 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:32.439815 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:32.439820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:32.440064 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:32.440936 (+ 872us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:10:32.440939 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:32.440999 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:10:32.441107 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:32.441466 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:32.441477 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:32.441486 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 15:10:32.441641 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 15:10:32.442035 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":151,"threads_started":1}
I20260807 15:10:32.444542 29996 heartbeater.cc:347] Connected to a master server at 127.15.226.126:38649
I20260807 15:10:32.444821 29996 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:32.445416 29996 heartbeater.cc:511] Master 127.15.226.126:38649 requested a full tablet report, sending...
I20260807 15:10:32.446583 29335 ts_manager.cc:194] Registered new tserver with Master: 43256b73ddf6405b9c63104c2ad4a4cd (127.15.226.67:35115)
I20260807 15:10:32.447280 29335 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.226.67:59345
I20260807 15:10:32.455885 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 15:10:32.459666 29389 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:32.458315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53834 (local address 127.15.226.126:38649)
0807 15:10:32.458494 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:32.458499 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:32.458834 (+ 335us) server_negotiation.cc:407] Connection header received
0807 15:10:32.458994 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:32.458997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:32.459046 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:32.459118 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:32.459492 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53834: BlockingRecv error: recv got EOF from 127.0.0.1:53834 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 15:10:32.459872 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 29422
I20260807 15:10:32.475677 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 29622
I20260807 15:10:32.492282 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 29811
I20260807 15:10:32.509380 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 29288
2026-08-07T15:10:32Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (1059 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T15:10:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:32Z Disabled control of system clock
I20260807 15:10:32.551553 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:46349
--webserver_interface=127.15.226.126
--webserver_port=0
--builtin_ntp_servers=127.15.226.84:39027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:46349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 15:10:32.671564 30015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:32.671891 30015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:32.671931 30015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:32.676025 30015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:32.676105 30015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:32.676121 30015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:32.676141 30015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:32.676156 30015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:32.681120 30015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:39027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:46349
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:46349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:32.682302 30015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:32.683836 30015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:32.692982 30026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:32.693056 30021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:32.693056 30022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:32.693466 30015 server_base.cc:1034] running on GCE node
I20260807 15:10:32.694105 30015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:32.695415 30015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:32.696592 30015 hybrid_clock.cc:648] HybridClock initialized: now 1786115432696577 us; error 44 us; skew 500 ppm
I20260807 15:10:32.698851 30015 webserver.cc:460] Webserver started at http://127.15.226.126:35733/ using document root <none> and password file <none>
I20260807 15:10:32.699491 30015 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:32.699561 30015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:32.699822 30015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:32.702080 30015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "058933b73a254b0188d8ebf20cb86b95"
format_stamp: "Formatted at 2026-08-07 15:10:32 on dist-test-slave-3kk6"
server_key: "4409f65359bcca711cca6eaf052d814a"
server_key_iv: "30aa98bf6b631e11a721c581faa32a51"
server_key_version: "encryptionkey@0"
I20260807 15:10:32.702767 30015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "058933b73a254b0188d8ebf20cb86b95"
format_stamp: "Formatted at 2026-08-07 15:10:32 on dist-test-slave-3kk6"
server_key: "4409f65359bcca711cca6eaf052d814a"
server_key_iv: "30aa98bf6b631e11a721c581faa32a51"
server_key_version: "encryptionkey@0"
I20260807 15:10:32.706867 30015 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:10:32.709645 30034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:32.711140 30015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:10:32.711298 30015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "058933b73a254b0188d8ebf20cb86b95"
format_stamp: "Formatted at 2026-08-07 15:10:32 on dist-test-slave-3kk6"
server_key: "4409f65359bcca711cca6eaf052d814a"
server_key_iv: "30aa98bf6b631e11a721c581faa32a51"
server_key_version: "encryptionkey@0"
I20260807 15:10:32.711422 30015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:32.716647 30015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:32.717342 30015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:32.717548 30015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:32.725347 30015 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:46349
I20260807 15:10:32.725366 30107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:46349 every 8 connection(s)
I20260807 15:10:32.726416 30015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:32.728793 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 30015
I20260807 15:10:32.728936 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:10:32.729262 16265 external_mini_cluster.cc:1428] Setting key 6e23dc797396e05b36e044852f07ab60
I20260807 15:10:32.730526 30108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:32.735459 30112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:32.730944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47628 (local address 127.15.226.126:46349)
0807 15:10:32.731632 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:32.731648 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:32.731686 (+ 38us) server_negotiation.cc:407] Connection header received
0807 15:10:32.732373 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:32.732389 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:32.732746 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:32.733159 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:32.734061 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:32.734066 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:32.734078 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:32.734088 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:32.734144 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:32.734160 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:32.734276 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:32.734408 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 15:10:32.734749 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":153,"threads_started":1}
I20260807 15:10:32.737237 30108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 058933b73a254b0188d8ebf20cb86b95: Bootstrap starting.
I20260807 15:10:32.739653 30108 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 058933b73a254b0188d8ebf20cb86b95: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:32.740442 30108 log.cc:824] T 00000000000000000000000000000000 P 058933b73a254b0188d8ebf20cb86b95: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:32.742632 30108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 058933b73a254b0188d8ebf20cb86b95: No bootstrap required, opened a new log
I20260807 15:10:32.746897 30108 raft_consensus.cc:348] T 00000000000000000000000000000000 P 058933b73a254b0188d8ebf20cb86b95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "058933b73a254b0188d8ebf20cb86b95" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46349 } }
I20260807 15:10:32.747184 30108 raft_consensus.cc:374] T 00000000000000000000000000000000 P 058933b73a254b0188d8ebf20cb86b95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:32.747223 30108 raft_consensus.cc:729] T 00000000000000000000000000000000 P 058933b73a254b0188d8ebf20cb86b95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 058933b73a254b0188d8ebf20cb86b95, State: Initialized, Role: FOLLOWER
I20260807 15:10:32.747680 30108 consensus_queue.cc:260] T 00000000000000000000000000000000 P 058933b73a254b0188d8ebf20cb86b95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "058933b73a254b0188d8ebf20cb86b95" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46349 } }
I20260807 15:10:32.747835 30108 raft_consensus.cc:388] T 00000000000000000000000000000000 P 058933b73a254b0188d8ebf20cb86b95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:32.747889 30108 raft_consensus.cc:482] T 00000000000000000000000000000000 P 058933b73a254b0188d8ebf20cb86b95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:32.747965 30108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 058933b73a254b0188d8ebf20cb86b95 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:32.749084 30108 raft_consensus.cc:504] T 00000000000000000000000000000000 P 058933b73a254b0188d8ebf20cb86b95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "058933b73a254b0188d8ebf20cb86b95" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46349 } }
I20260807 15:10:32.749449 30108 leader_election.cc:302] T 00000000000000000000000000000000 P 058933b73a254b0188d8ebf20cb86b95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 058933b73a254b0188d8ebf20cb86b95; no voters:
I20260807 15:10:32.749858 30108 leader_election.cc:288] T 00000000000000000000000000000000 P 058933b73a254b0188d8ebf20cb86b95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:10:32.749997 30116 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 058933b73a254b0188d8ebf20cb86b95 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:32.750313 30116 raft_consensus.cc:686] T 00000000000000000000000000000000 P 058933b73a254b0188d8ebf20cb86b95 [term 1 LEADER]: Becoming Leader. State: Replica: 058933b73a254b0188d8ebf20cb86b95, State: Running, Role: LEADER
I20260807 15:10:32.750765 30116 consensus_queue.cc:237] T 00000000000000000000000000000000 P 058933b73a254b0188d8ebf20cb86b95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "058933b73a254b0188d8ebf20cb86b95" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46349 } }
I20260807 15:10:32.751338 30108 sys_catalog.cc:564] T 00000000000000000000000000000000 P 058933b73a254b0188d8ebf20cb86b95 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:32.752676 30119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 058933b73a254b0188d8ebf20cb86b95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 058933b73a254b0188d8ebf20cb86b95. Latest consensus state: current_term: 1 leader_uuid: "058933b73a254b0188d8ebf20cb86b95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "058933b73a254b0188d8ebf20cb86b95" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46349 } } }
I20260807 15:10:32.752676 30117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 058933b73a254b0188d8ebf20cb86b95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "058933b73a254b0188d8ebf20cb86b95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "058933b73a254b0188d8ebf20cb86b95" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46349 } } }
I20260807 15:10:32.753000 30117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 058933b73a254b0188d8ebf20cb86b95 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:32.753338 30119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 058933b73a254b0188d8ebf20cb86b95 [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:32.753515 30136 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:32.757007 30136 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:10:32.762547 30136 catalog_manager.cc:1347] Generated new cluster ID: 717dcc17a9ac4094a408c37a3724320f
I20260807 15:10:32.762635 30136 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:10:32.774616 30136 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:10:32.775027 30136 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:32.783568 30136 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 058933b73a254b0188d8ebf20cb86b95: Generated new TSK 0
I20260807 15:10:32.784464 30136 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:10:32.787833 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:46349
--builtin_ntp_servers=127.15.226.84:39027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 15:10:32.907871 30147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:32.908216 30147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:32.908309 30147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:32.912624 30147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:32.912730 30147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:32.912847 30147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:10:32.918066 30147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:39027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:46349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:32.919353 30147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:32.920817 30147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:32.931157 30159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:32.931160 30157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:32.931157 30155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:32.931488 30147 server_base.cc:1034] running on GCE node
I20260807 15:10:32.932049 30147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:32.932746 30147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:32.933960 30147 hybrid_clock.cc:648] HybridClock initialized: now 1786115432933929 us; error 62 us; skew 500 ppm
I20260807 15:10:32.936167 30147 webserver.cc:460] Webserver started at http://127.15.226.65:40557/ using document root <none> and password file <none>
I20260807 15:10:32.936830 30147 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:32.936936 30147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:32.937175 30147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:32.939572 30147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "9b9964fdb6ed4defbda279f15e3cdb7f"
format_stamp: "Formatted at 2026-08-07 15:10:32 on dist-test-slave-3kk6"
server_key: "533ded2b0375c00de681bb1ecf3adb9a"
server_key_iv: "3669c1bec43e20c9ac18d8ca9f9109a5"
server_key_version: "encryptionkey@0"
I20260807 15:10:32.940238 30147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "9b9964fdb6ed4defbda279f15e3cdb7f"
format_stamp: "Formatted at 2026-08-07 15:10:32 on dist-test-slave-3kk6"
server_key: "533ded2b0375c00de681bb1ecf3adb9a"
server_key_iv: "3669c1bec43e20c9ac18d8ca9f9109a5"
server_key_version: "encryptionkey@0"
I20260807 15:10:32.944399 30147 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:10:32.947295 30168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:32.948711 30147 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:10:32.948920 30147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "9b9964fdb6ed4defbda279f15e3cdb7f"
format_stamp: "Formatted at 2026-08-07 15:10:32 on dist-test-slave-3kk6"
server_key: "533ded2b0375c00de681bb1ecf3adb9a"
server_key_iv: "3669c1bec43e20c9ac18d8ca9f9109a5"
server_key_version: "encryptionkey@0"
I20260807 15:10:32.949064 30147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:32.954607 30147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:32.955399 30147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:32.955639 30147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:32.956255 30147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:32.957257 30147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:32.957330 30147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:32.957406 30147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:32.957450 30147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:32.966940 30147 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:34179
I20260807 15:10:32.966964 30338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:34179 every 8 connection(s)
I20260807 15:10:32.967952 30147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:10:32.974445 30112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:32.969929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:33721 (local address 127.15.226.126:46349)
0807 15:10:32.970119 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:32.970124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:32.971306 (+ 1182us) server_negotiation.cc:407] Connection header received
0807 15:10:32.972742 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:32.972745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:32.972786 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:32.972866 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:32.974035 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:32.974038 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:32.974040 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:32.974053 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:32.974073 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:32.974081 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:32.974169 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:32.974288 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 15:10:32.974327 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 15:10:32.975431 30342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:32.970369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46349 (local address 127.15.226.65:33721)
0807 15:10:32.971124 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:32.971177 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:32.972533 (+ 1356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:32.973037 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:32.973044 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:32.973135 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:32.973729 (+ 594us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:10:32.973731 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:32.973774 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:10:32.973846 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:32.974162 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:32.974169 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:32.974173 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:10:32.974311 (+ 138us) client_negotiation.cc:240] Negotiation successful
0807 15:10:32.974526 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":131,"threads_started":1}
I20260807 15:10:32.976193 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 30147
I20260807 15:10:32.976336 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:10:32.976647 16265 external_mini_cluster.cc:1428] Setting key 7917c701295fea27ccab9134e510f1b0
I20260807 15:10:32.976737 30340 heartbeater.cc:347] Connected to a master server at 127.15.226.126:46349
I20260807 15:10:32.976979 30340 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:32.977524 30340 heartbeater.cc:511] Master 127.15.226.126:46349 requested a full tablet report, sending...
I20260807 15:10:32.978868 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:46349
--builtin_ntp_servers=127.15.226.84:39027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 15:10:32.979214 30057 ts_manager.cc:194] Registered new tserver with Master: 9b9964fdb6ed4defbda279f15e3cdb7f (127.15.226.65:34179)
W20260807 15:10:33.098959 30343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:33.099349 30343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:33.099474 30343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:33.103817 30343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:33.103932 30343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:33.104038 30343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:10:33.109812 30343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:39027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:46349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:33.111102 30343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:33.112594 30343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:33.123246 30358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:33.123267 30359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:33.123251 30362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:33.123313 30343 server_base.cc:1034] running on GCE node
I20260807 15:10:33.124037 30343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:33.124806 30343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:33.126006 30343 hybrid_clock.cc:648] HybridClock initialized: now 1786115433125989 us; error 31 us; skew 500 ppm
I20260807 15:10:33.131332 30343 webserver.cc:460] Webserver started at http://127.15.226.66:33039/ using document root <none> and password file <none>
I20260807 15:10:33.132020 30343 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:33.132138 30343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:33.132381 30343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:33.134855 30343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "d354cfc94d0145dea55b7c3d58879925"
format_stamp: "Formatted at 2026-08-07 15:10:33 on dist-test-slave-3kk6"
server_key: "e9e1f0742f21077a874c879e789fd936"
server_key_iv: "01d515fdf4474ff42e5e7f56aedc3607"
server_key_version: "encryptionkey@0"
I20260807 15:10:33.135501 30343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "d354cfc94d0145dea55b7c3d58879925"
format_stamp: "Formatted at 2026-08-07 15:10:33 on dist-test-slave-3kk6"
server_key: "e9e1f0742f21077a874c879e789fd936"
server_key_iv: "01d515fdf4474ff42e5e7f56aedc3607"
server_key_version: "encryptionkey@0"
I20260807 15:10:33.140040 30343 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 15:10:33.142881 30369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:33.144148 30343 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260807 15:10:33.144304 30343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "d354cfc94d0145dea55b7c3d58879925"
format_stamp: "Formatted at 2026-08-07 15:10:33 on dist-test-slave-3kk6"
server_key: "e9e1f0742f21077a874c879e789fd936"
server_key_iv: "01d515fdf4474ff42e5e7f56aedc3607"
server_key_version: "encryptionkey@0"
I20260807 15:10:33.144441 30343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:33.149556 30343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:33.150245 30343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:33.150456 30343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:33.151204 30343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:33.152324 30343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:33.152400 30343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:33.152472 30343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:33.152526 30343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:33.162420 30343 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:35165
I20260807 15:10:33.162452 30541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:35165 every 8 connection(s)
I20260807 15:10:33.163491 30343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:10:33.167097 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 30343
I20260807 15:10:33.167223 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:10:33.167557 16265 external_mini_cluster.cc:1428] Setting key c3cbda5e050b2d50ad66adb452b5f31c
I20260807 15:10:33.170032 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:46349
--builtin_ntp_servers=127.15.226.84:39027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 15:10:33.170549 30112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:33.165368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:47405 (local address 127.15.226.126:46349)
0807 15:10:33.165563 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:33.165568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:33.166696 (+ 1128us) server_negotiation.cc:407] Connection header received
0807 15:10:33.168556 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:33.168559 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:33.168596 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:33.168668 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:33.170037 (+ 1369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:33.170040 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:33.170042 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:33.170054 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:33.170075 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:33.170083 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:33.170207 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:33.170388 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 15:10:33.170440 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 15:10:33.171775 30545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:33.165802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46349 (local address 127.15.226.66:47405)
0807 15:10:33.166495 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:33.166537 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:33.168311 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:33.168874 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:33.168883 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:33.169005 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:33.169680 (+ 675us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:10:33.169682 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:33.169743 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:10:33.169866 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:33.170213 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:33.170227 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:33.170237 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 15:10:33.170427 (+ 190us) client_negotiation.cc:240] Negotiation successful
0807 15:10:33.170716 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":127,"threads_started":1}
I20260807 15:10:33.172986 30542 heartbeater.cc:347] Connected to a master server at 127.15.226.126:46349
I20260807 15:10:33.173254 30542 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:33.173780 30542 heartbeater.cc:511] Master 127.15.226.126:46349 requested a full tablet report, sending...
I20260807 15:10:33.174955 30057 ts_manager.cc:194] Registered new tserver with Master: d354cfc94d0145dea55b7c3d58879925 (127.15.226.66:35165)
W20260807 15:10:33.289750 30546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:33.290158 30546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:33.290246 30546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:33.294515 30546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:33.294641 30546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:33.294767 30546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:10:33.299867 30546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:39027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:46349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:33.301086 30546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:33.302582 30546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:33.313280 30554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:33.314265 30546 server_base.cc:1034] running on GCE node
W20260807 15:10:33.314276 30555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:33.314285 30559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:33.314950 30546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:33.315768 30546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:33.316941 30546 hybrid_clock.cc:648] HybridClock initialized: now 1786115433316920 us; error 38 us; skew 500 ppm
I20260807 15:10:33.319155 30546 webserver.cc:460] Webserver started at http://127.15.226.67:41493/ using document root <none> and password file <none>
I20260807 15:10:33.319749 30546 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:33.319823 30546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:33.320001 30546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:33.322150 30546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "0079ac07840d4c298257b33939aa0b00"
format_stamp: "Formatted at 2026-08-07 15:10:33 on dist-test-slave-3kk6"
server_key: "d6ebf8e13d33538b5c2c4b402b49143b"
server_key_iv: "306fed0c64fc8c1bed50ed147ca0893c"
server_key_version: "encryptionkey@0"
I20260807 15:10:33.322832 30546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "0079ac07840d4c298257b33939aa0b00"
format_stamp: "Formatted at 2026-08-07 15:10:33 on dist-test-slave-3kk6"
server_key: "d6ebf8e13d33538b5c2c4b402b49143b"
server_key_iv: "306fed0c64fc8c1bed50ed147ca0893c"
server_key_version: "encryptionkey@0"
I20260807 15:10:33.326967 30546 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:10:33.329703 30568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:33.331176 30546 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:10:33.331329 30546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "0079ac07840d4c298257b33939aa0b00"
format_stamp: "Formatted at 2026-08-07 15:10:33 on dist-test-slave-3kk6"
server_key: "d6ebf8e13d33538b5c2c4b402b49143b"
server_key_iv: "306fed0c64fc8c1bed50ed147ca0893c"
server_key_version: "encryptionkey@0"
I20260807 15:10:33.331466 30546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:33.336433 30546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:33.337124 30546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:33.337332 30546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:33.337934 30546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:33.338979 30546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:33.339051 30546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:33.339125 30546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:33.339179 30546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:33.348515 30546 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:43725
I20260807 15:10:33.348553 30742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:43725 every 8 connection(s)
I20260807 15:10:33.349570 30546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:10:33.357319 30112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:33.351686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:42115 (local address 127.15.226.126:46349)
0807 15:10:33.351869 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:33.351875 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:33.353790 (+ 1915us) server_negotiation.cc:407] Connection header received
0807 15:10:33.355416 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:33.355421 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:33.355474 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:33.355561 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:33.356794 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:33.356798 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:33.356801 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:10:33.356815 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:33.356843 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:10:33.356852 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:33.356967 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:33.357115 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 15:10:33.357168 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 15:10:33.358232 30749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:33.352195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46349 (local address 127.15.226.67:42115)
0807 15:10:33.353622 (+ 1427us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:33.353669 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:33.355165 (+ 1496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:33.355744 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:33.355751 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:33.355845 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:33.356501 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:10:33.356503 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:33.356546 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:10:33.356619 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:33.356940 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:33.356950 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:33.356956 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:10:33.357162 (+ 206us) client_negotiation.cc:240] Negotiation successful
0807 15:10:33.357422 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":912,"thread_start_us":201,"threads_started":1}
I20260807 15:10:33.359583 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 30546
I20260807 15:10:33.359678 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:10:33.359758 30745 heartbeater.cc:347] Connected to a master server at 127.15.226.126:46349
I20260807 15:10:33.360081 30745 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:33.360095 16265 external_mini_cluster.cc:1428] Setting key fcc1d2cb171979a17606616a01633e11
I20260807 15:10:33.360548 30745 heartbeater.cc:511] Master 127.15.226.126:46349 requested a full tablet report, sending...
I20260807 15:10:33.361764 30057 ts_manager.cc:194] Registered new tserver with Master: 0079ac07840d4c298257b33939aa0b00 (127.15.226.67:43725)
I20260807 15:10:33.362495 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 15:10:33.365451 30112 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:10:33.364606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47644 (local address 127.15.226.126:46349)
0807 15:10:33.364759 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:33.364763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:33.364927 (+ 164us) server_negotiation.cc:407] Connection header received
0807 15:10:33.365022 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:33.365024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:33.365053 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:33.365121 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:33.365304 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47644: BlockingRecv error: recv got EOF from 127.0.0.1:47644 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 15:10:33.365916 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 30147
I20260807 15:10:33.377049 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 30343
I20260807 15:10:33.409500 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 30546
I20260807 15:10:33.419415 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 30015
2026-08-07T15:10:33Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (906 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:10:33 dist-test-slave-3kk6 krb5kdc[30760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:10:33 dist-test-slave-3kk6 krb5kdc[30760](info): set up 2 sockets
Aug 07 15:10:33 dist-test-slave-3kk6 krb5kdc[30760](info): commencing operation
krb5kdc: starting...
W20260807 15:10:34.694259 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:10:34 dist-test-slave-3kk6 krb5kdc[30760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:10:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:10:34.837924 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:36387
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:41495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:36387
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:34.959021 30776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:34.959415 30776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:34.959496 30776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:34.963791 30776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:10:34.963938 30776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:34.963979 30776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:34.964022 30776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:10:34.964059 30776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:10:34.969887 30776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:41495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:36387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:36387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:34.971287 30776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:34.972525 30776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:10:34.982364 30776 server_base.cc:1034] running on GCE node
W20260807 15:10:34.982314 30792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:34.982300 30788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:34.982321 30790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:34.983333 30776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:34.984706 30776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:34.985961 30776 hybrid_clock.cc:648] HybridClock initialized: now 1786115434985941 us; error 41 us; skew 500 ppm
Aug 07 15:10:34 dist-test-slave-3kk6 krb5kdc[30760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115434, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:34.989322 30776 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:10:34.990337 30776 webserver.cc:460] Webserver started at http://127.15.226.126:45933/ using document root <none> and password file <none>
I20260807 15:10:34.991082 30776 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:34.991186 30776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:34.991446 30776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:34.993733 30776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "eb7e03e0b93142fcaecc9b358e18ebea"
format_stamp: "Formatted at 2026-08-07 15:10:34 on dist-test-slave-3kk6"
server_key: "f78e63df89ae883a9080021a9ae4a5ec"
server_key_iv: "7b9cf29bd46d531a8b46ebb6713ca43f"
server_key_version: "encryptionkey@0"
I20260807 15:10:34.994385 30776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "eb7e03e0b93142fcaecc9b358e18ebea"
format_stamp: "Formatted at 2026-08-07 15:10:34 on dist-test-slave-3kk6"
server_key: "f78e63df89ae883a9080021a9ae4a5ec"
server_key_iv: "7b9cf29bd46d531a8b46ebb6713ca43f"
server_key_version: "encryptionkey@0"
I20260807 15:10:34.998632 30776 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 15:10:35.001505 30804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:35.003234 30776 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 15:10:35.003422 30776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "eb7e03e0b93142fcaecc9b358e18ebea"
format_stamp: "Formatted at 2026-08-07 15:10:34 on dist-test-slave-3kk6"
server_key: "f78e63df89ae883a9080021a9ae4a5ec"
server_key_iv: "7b9cf29bd46d531a8b46ebb6713ca43f"
server_key_version: "encryptionkey@0"
I20260807 15:10:35.003575 30776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:35.017247 30776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:35.021083 30776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:35.021332 30776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:35.028709 30776 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:36387
I20260807 15:10:35.029040 30891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:36387 every 8 connection(s)
I20260807 15:10:35.029847 30776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:10:35.033000 30892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:35.037122 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 30776
I20260807 15:10:35.037245 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:10:35.037561 16265 external_mini_cluster.cc:1428] Setting key dda449f5a384a210baaa2830b0ce8fc6
I20260807 15:10:35.039535 30892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb7e03e0b93142fcaecc9b358e18ebea: Bootstrap starting.
I20260807 15:10:35.042271 30892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb7e03e0b93142fcaecc9b358e18ebea: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:35.043200 30892 log.cc:824] T 00000000000000000000000000000000 P eb7e03e0b93142fcaecc9b358e18ebea: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:35.045667 30892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb7e03e0b93142fcaecc9b358e18ebea: No bootstrap required, opened a new log
Aug 07 15:10:35 dist-test-slave-3kk6 krb5kdc[30760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:10:35.049839 30892 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb7e03e0b93142fcaecc9b358e18ebea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb7e03e0b93142fcaecc9b358e18ebea" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36387 } }
I20260807 15:10:35.050133 30892 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb7e03e0b93142fcaecc9b358e18ebea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:35.050220 30892 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb7e03e0b93142fcaecc9b358e18ebea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb7e03e0b93142fcaecc9b358e18ebea, State: Initialized, Role: FOLLOWER
I20260807 15:10:35.050808 30892 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb7e03e0b93142fcaecc9b358e18ebea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb7e03e0b93142fcaecc9b358e18ebea" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36387 } }
I20260807 15:10:35.050994 30892 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb7e03e0b93142fcaecc9b358e18ebea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:10:35.051069 30892 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb7e03e0b93142fcaecc9b358e18ebea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:10:35.051182 30892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb7e03e0b93142fcaecc9b358e18ebea [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:35.052254 30892 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb7e03e0b93142fcaecc9b358e18ebea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb7e03e0b93142fcaecc9b358e18ebea" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36387 } }
I20260807 15:10:35.052634 30892 leader_election.cc:302] T 00000000000000000000000000000000 P eb7e03e0b93142fcaecc9b358e18ebea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb7e03e0b93142fcaecc9b358e18ebea; no voters:
I20260807 15:10:35.053020 30892 leader_election.cc:288] T 00000000000000000000000000000000 P eb7e03e0b93142fcaecc9b358e18ebea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:10:35.053128 30902 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb7e03e0b93142fcaecc9b358e18ebea [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:35.053354 30902 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb7e03e0b93142fcaecc9b358e18ebea [term 1 LEADER]: Becoming Leader. State: Replica: eb7e03e0b93142fcaecc9b358e18ebea, State: Running, Role: LEADER
I20260807 15:10:35.053679 30902 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb7e03e0b93142fcaecc9b358e18ebea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb7e03e0b93142fcaecc9b358e18ebea" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36387 } }
I20260807 15:10:35.053920 30899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:35.039162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48684 (local address 127.15.226.126:36387)
0807 15:10:35.039623 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:35.039641 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:35.039717 (+ 76us) server_negotiation.cc:407] Connection header received
0807 15:10:35.040461 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:35.040481 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:35.040838 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:35.041147 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:35.042181 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.043437 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.044507 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.045011 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.045140 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:35.048109 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:35.048115 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:35.048131 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:35.048163 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:35.050121 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:35.050752 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:35.050757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:35.050762 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:35.050888 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:35.051243 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:35.051251 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:35.051252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:35.051768 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:35.052028 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:35.052274 (+ 246us) server_negotiation.cc:299] Negotiation successful
0807 15:10:35.052736 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":146,"threads_started":1}
I20260807 15:10:35.054395 30892 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb7e03e0b93142fcaecc9b358e18ebea [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:10:35.054970 30903 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb7e03e0b93142fcaecc9b358e18ebea [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb7e03e0b93142fcaecc9b358e18ebea. Latest consensus state: current_term: 1 leader_uuid: "eb7e03e0b93142fcaecc9b358e18ebea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb7e03e0b93142fcaecc9b358e18ebea" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36387 } } }
I20260807 15:10:35.055214 30903 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb7e03e0b93142fcaecc9b358e18ebea [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:35.055510 30904 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb7e03e0b93142fcaecc9b358e18ebea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb7e03e0b93142fcaecc9b358e18ebea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb7e03e0b93142fcaecc9b358e18ebea" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36387 } } }
I20260807 15:10:35.055629 30904 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb7e03e0b93142fcaecc9b358e18ebea [sys.catalog]: This master's current role is: LEADER
I20260807 15:10:35.056685 30910 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:10:35.059861 30910 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:10:35.066226 30910 catalog_manager.cc:1347] Generated new cluster ID: 9e0c217dc0aa46a29dd7524d901fdeea
I20260807 15:10:35.066336 30910 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:10:35.073263 30910 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:10:35.074419 30910 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:10:35.082448 30910 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb7e03e0b93142fcaecc9b358e18ebea: Generated new TSK 0
I20260807 15:10:35.083457 30910 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.226.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
I20260807 15:10:35.122269 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:36387
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:41495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:35.246876 30936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:35.247201 30936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:35.247251 30936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:35.251498 30936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:35.251572 30936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:35.251634 30936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:10:35.257388 30936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:41495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:36387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:35.258531 30936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:35.259647 30936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:35.270394 30945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:35.270412 30942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:35.270570 30936 server_base.cc:1034] running on GCE node
W20260807 15:10:35.270903 30947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:35.271359 30936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:35.272104 30936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:35.273296 30936 hybrid_clock.cc:648] HybridClock initialized: now 1786115435273277 us; error 38 us; skew 500 ppm
Aug 07 15:10:35 dist-test-slave-3kk6 krb5kdc[30760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115435, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:35.276297 30936 init.cc:377] Logged in from keytab as kudu/127.15.226.65@KRBTEST.COM (short username kudu)
I20260807 15:10:35.277266 30936 webserver.cc:460] Webserver started at http://127.15.226.65:33865/ using document root <none> and password file <none>
I20260807 15:10:35.277809 30936 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:35.277875 30936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:35.278046 30936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:35.280215 30936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "ca7162ce47e640ba9a74c2c1b853e879"
format_stamp: "Formatted at 2026-08-07 15:10:35 on dist-test-slave-3kk6"
server_key: "ffa9f850ec48877ea8a823c975826935"
server_key_iv: "8135be0c03d257348c43fee58a9ca596"
server_key_version: "encryptionkey@0"
I20260807 15:10:35.280802 30936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "ca7162ce47e640ba9a74c2c1b853e879"
format_stamp: "Formatted at 2026-08-07 15:10:35 on dist-test-slave-3kk6"
server_key: "ffa9f850ec48877ea8a823c975826935"
server_key_iv: "8135be0c03d257348c43fee58a9ca596"
server_key_version: "encryptionkey@0"
I20260807 15:10:35.284763 30936 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:10:35.287432 30959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:35.288779 30936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:10:35.288928 30936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "ca7162ce47e640ba9a74c2c1b853e879"
format_stamp: "Formatted at 2026-08-07 15:10:35 on dist-test-slave-3kk6"
server_key: "ffa9f850ec48877ea8a823c975826935"
server_key_iv: "8135be0c03d257348c43fee58a9ca596"
server_key_version: "encryptionkey@0"
I20260807 15:10:35.289062 30936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:35.324369 30936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:35.328109 30936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:35.328361 30936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:35.329082 30936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:35.330040 30936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:35.330092 30936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:35.330137 30936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:35.330152 30936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:35.340849 30936 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:42483
I20260807 15:10:35.340870 31138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:42483 every 8 connection(s)
I20260807 15:10:35.341934 30936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:10:35.351850 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 30936
I20260807 15:10:35.352038 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:10:35.352351 16265 external_mini_cluster.cc:1428] Setting key d583d27ac662ad54828209e35fa8431f
Aug 07 15:10:35 dist-test-slave-3kk6 krb5kdc[30760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115435, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:10:35.359385 30899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:35.344057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:45009 (local address 127.15.226.126:36387)
0807 15:10:35.344219 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:35.344223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:35.345126 (+ 903us) server_negotiation.cc:407] Connection header received
0807 15:10:35.346915 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:35.346919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:35.346990 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:35.347115 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:35.348536 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.349167 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.350491 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.350852 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.350986 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:35.354332 (+ 3346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:35.354336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:35.354338 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:35.354365 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:35.356458 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:35.357381 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:35.357383 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:35.357385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:35.357483 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:35.358000 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:35.358002 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:35.358003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:35.358196 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:35.358273 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:35.359106 (+ 833us) server_negotiation.cc:299] Negotiation successful
0807 15:10:35.359248 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 15:10:35.360978 31142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:35.344410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36387 (local address 127.15.226.65:45009)
0807 15:10:35.344924 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:35.344974 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:35.346594 (+ 1620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:35.347332 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:35.347339 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:35.347780 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:35.348346 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:35.348363 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.349328 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.349332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:35.350337 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:35.350348 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.351041 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.351046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:35.351246 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:35.352061 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:35.352087 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:35.354124 (+ 2037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:35.356633 (+ 2509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:35.356648 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:35.356670 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:35.357252 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:35.357594 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:35.357601 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:35.357606 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:35.357880 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:35.358319 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:35.358330 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:35.358799 (+ 469us) client_negotiation.cc:769] Sending connection context
0807 15:10:35.359075 (+ 276us) client_negotiation.cc:240] Negotiation successful
0807 15:10:35.359536 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":115,"threads_started":1}
I20260807 15:10:35.362530 31139 heartbeater.cc:347] Connected to a master server at 127.15.226.126:36387
I20260807 15:10:35.362829 31139 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:35.363384 31139 heartbeater.cc:511] Master 127.15.226.126:36387 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.226.66@KRBTEST.COM; defaulting to no policy
I20260807 15:10:35.365275 30828 ts_manager.cc:194] Registered new tserver with Master: ca7162ce47e640ba9a74c2c1b853e879 (127.15.226.65:42483)
I20260807 15:10:35.366830 30828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.65@KRBTEST.COM'} at 127.15.226.65:45009
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
I20260807 15:10:35.386200 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:36387
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:41495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:35.504848 31151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:35.505159 31151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:35.505208 31151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:35.509691 31151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:35.509809 31151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:35.509927 31151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:10:35.515838 31151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:41495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:36387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:35.517009 31151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:35.518086 31151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:35.528877 31162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:35.528916 31151 server_base.cc:1034] running on GCE node
W20260807 15:10:35.528892 31160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:35.528872 31157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:35.529631 31151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:35.530316 31151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:35.531505 31151 hybrid_clock.cc:648] HybridClock initialized: now 1786115435531491 us; error 27 us; skew 500 ppm
Aug 07 15:10:35 dist-test-slave-3kk6 krb5kdc[30760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115435, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:35.534812 31151 init.cc:377] Logged in from keytab as kudu/127.15.226.66@KRBTEST.COM (short username kudu)
I20260807 15:10:35.535748 31151 webserver.cc:460] Webserver started at http://127.15.226.66:37519/ using document root <none> and password file <none>
I20260807 15:10:35.536350 31151 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:35.536459 31151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:35.536715 31151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:35.539033 31151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "2115efa6ea79429cb0f72e4855ad03e1"
format_stamp: "Formatted at 2026-08-07 15:10:35 on dist-test-slave-3kk6"
server_key: "b6937161f94c34b9d5ca8b29348964df"
server_key_iv: "47cab75fc3ed34faa7cc004a27189175"
server_key_version: "encryptionkey@0"
I20260807 15:10:35.539654 31151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "2115efa6ea79429cb0f72e4855ad03e1"
format_stamp: "Formatted at 2026-08-07 15:10:35 on dist-test-slave-3kk6"
server_key: "b6937161f94c34b9d5ca8b29348964df"
server_key_iv: "47cab75fc3ed34faa7cc004a27189175"
server_key_version: "encryptionkey@0"
I20260807 15:10:35.543815 31151 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:10:35.546557 31174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:35.548030 31151 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:10:35.548188 31151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "2115efa6ea79429cb0f72e4855ad03e1"
format_stamp: "Formatted at 2026-08-07 15:10:35 on dist-test-slave-3kk6"
server_key: "b6937161f94c34b9d5ca8b29348964df"
server_key_iv: "47cab75fc3ed34faa7cc004a27189175"
server_key_version: "encryptionkey@0"
I20260807 15:10:35.548331 31151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:35.562467 31151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:35.565846 31151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:35.566078 31151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:35.566792 31151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:35.567670 31151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:35.567752 31151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:35.567858 31151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:35.567911 31151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:35.577418 31151 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:41949
I20260807 15:10:35.577448 31335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:41949 every 8 connection(s)
I20260807 15:10:35.578353 31151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:10:35.584110 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 31151
I20260807 15:10:35.584250 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:10:35.584652 16265 external_mini_cluster.cc:1428] Setting key 9cb95b4bd3661e93ffe0a1031ea34ef5
Aug 07 15:10:35 dist-test-slave-3kk6 krb5kdc[30760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115435, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:10:35.594273 30899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:35.580331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:38503 (local address 127.15.226.126:36387)
0807 15:10:35.580475 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:35.580478 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:35.581380 (+ 902us) server_negotiation.cc:407] Connection header received
0807 15:10:35.582983 (+ 1603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:35.582985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:35.583037 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:35.583119 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:35.584602 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.585302 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.586739 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.586977 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.587091 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:35.590144 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:35.590147 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:35.590150 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:35.590177 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:35.591917 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:35.592503 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:35.592508 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:35.592510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:35.592598 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:35.593030 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:35.593033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:35.593035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:35.593245 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:35.593359 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:35.593943 (+ 584us) server_negotiation.cc:299] Negotiation successful
0807 15:10:35.594089 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:10:35.595542 31340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:35.580733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36387 (local address 127.15.226.66:38503)
0807 15:10:35.581196 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:35.581243 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:35.582757 (+ 1514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:35.583360 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:35.583368 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:35.583776 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:35.584423 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:35.584441 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.585450 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.585455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:35.586561 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:35.586573 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.587159 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.587163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:35.587353 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:35.588049 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:35.588066 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:35.589942 (+ 1876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:35.592055 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:35.592062 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:35.592088 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:35.592375 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:35.592698 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:35.592701 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:35.592703 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:35.592925 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:35.593361 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:35.593371 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:35.593664 (+ 293us) client_negotiation.cc:769] Sending connection context
0807 15:10:35.593877 (+ 213us) client_negotiation.cc:240] Negotiation successful
0807 15:10:35.594243 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":121,"threads_started":1}
WARNING: no policy specified for kudu/127.15.226.67@KRBTEST.COM; defaulting to no policy
I20260807 15:10:35.596988 31338 heartbeater.cc:347] Connected to a master server at 127.15.226.126:36387
I20260807 15:10:35.597280 31338 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:35.597903 31338 heartbeater.cc:511] Master 127.15.226.126:36387 requested a full tablet report, sending...
I20260807 15:10:35.599223 30828 ts_manager.cc:194] Registered new tserver with Master: 2115efa6ea79429cb0f72e4855ad03e1 (127.15.226.66:41949)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.67@KRBTEST.COM" created.
I20260807 15:10:35.599896 30828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.66@KRBTEST.COM'} at 127.15.226.66:38503
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
I20260807 15:10:35.617607 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:36387
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:41495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:10:35.737972 31343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:10:35.738276 31343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:10:35.738320 31343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:10:35.742727 31343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:10:35.742805 31343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:10:35.742867 31343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:10:35.748526 31343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:41495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:36387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:10:35.749643 31343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:10:35.750756 31343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:10:35.761667 31351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:35.761677 31355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:10:35.761694 31352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:10:35.762380 31343 server_base.cc:1034] running on GCE node
I20260807 15:10:35.762910 31343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:10:35.763618 31343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:10:35.764820 31343 hybrid_clock.cc:648] HybridClock initialized: now 1786115435764775 us; error 62 us; skew 500 ppm
Aug 07 15:10:35 dist-test-slave-3kk6 krb5kdc[30760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115435, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:10:35.767939 31343 init.cc:377] Logged in from keytab as kudu/127.15.226.67@KRBTEST.COM (short username kudu)
I20260807 15:10:35.769047 31343 webserver.cc:460] Webserver started at http://127.15.226.67:40883/ using document root <none> and password file <none>
I20260807 15:10:35.769658 31343 fs_manager.cc:359] Metadata directory not provided
I20260807 15:10:35.769758 31343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:10:35.770007 31343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:10:35.772281 31343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "df82f490d9c84676bdbe2a814347a678"
format_stamp: "Formatted at 2026-08-07 15:10:35 on dist-test-slave-3kk6"
server_key: "836cbe66a109c4efae5f93e9a07bf3f5"
server_key_iv: "2a0e3150b0aa5be8d2e3245dfde09380"
server_key_version: "encryptionkey@0"
I20260807 15:10:35.772939 31343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "df82f490d9c84676bdbe2a814347a678"
format_stamp: "Formatted at 2026-08-07 15:10:35 on dist-test-slave-3kk6"
server_key: "836cbe66a109c4efae5f93e9a07bf3f5"
server_key_iv: "2a0e3150b0aa5be8d2e3245dfde09380"
server_key_version: "encryptionkey@0"
I20260807 15:10:35.777071 31343 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:10:35.779740 31365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:35.781018 31343 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:10:35.781124 31343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "df82f490d9c84676bdbe2a814347a678"
format_stamp: "Formatted at 2026-08-07 15:10:35 on dist-test-slave-3kk6"
server_key: "836cbe66a109c4efae5f93e9a07bf3f5"
server_key_iv: "2a0e3150b0aa5be8d2e3245dfde09380"
server_key_version: "encryptionkey@0"
I20260807 15:10:35.781221 31343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:10:35.829068 31343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:10:35.832460 31343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:10:35.832705 31343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:10:35.833396 31343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:10:35.834317 31343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:10:35.834367 31343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:35.834440 31343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:10:35.834481 31343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:10:35.844198 31343 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:39137
I20260807 15:10:35.844229 31544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:39137 every 8 connection(s)
I20260807 15:10:35.845213 31343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:10:35.846603 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 31343
I20260807 15:10:35.846760 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:10:35.847095 16265 external_mini_cluster.cc:1428] Setting key a946944c8b23eec58475b9c38a51d9df
Aug 07 15:10:35 dist-test-slave-3kk6 krb5kdc[30760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115435, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:10:35.860630 30899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:35.847454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:49245 (local address 127.15.226.126:36387)
0807 15:10:35.847616 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:35.847620 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:35.848678 (+ 1058us) server_negotiation.cc:407] Connection header received
0807 15:10:35.850203 (+ 1525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:35.850221 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:35.850270 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:35.850351 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:35.851760 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.852433 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.853703 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.853928 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.854017 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:35.856932 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:35.856935 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:35.856937 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:35.856964 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:35.858371 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:35.859010 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:35.859013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:35.859014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:35.859095 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:35.859427 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:35.859429 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:35.859430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:35.859632 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:35.859710 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:35.860320 (+ 610us) server_negotiation.cc:299] Negotiation successful
0807 15:10:35.860490 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 15:10:35.861763 31551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:35.847954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36387 (local address 127.15.226.67:49245)
0807 15:10:35.848533 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:35.848591 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:35.849962 (+ 1371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:35.850539 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:35.850546 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:35.851061 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:10:35.851590 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:35.851605 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.852559 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.852562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:35.853586 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:35.853597 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.854076 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.854079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:35.854225 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:35.854994 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:10:35.855012 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:10:35.856746 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:10:35.858503 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:35.858509 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:35.858523 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:35.858906 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:35.859182 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:10:35.859184 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:10:35.859185 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:10:35.859334 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:10:35.859749 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:10:35.859759 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:10:35.860056 (+ 297us) client_negotiation.cc:769] Sending connection context
0807 15:10:35.860227 (+ 171us) client_negotiation.cc:240] Negotiation successful
0807 15:10:35.860664 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":171,"threads_started":1}
I20260807 15:10:35.862960 31545 heartbeater.cc:347] Connected to a master server at 127.15.226.126:36387
I20260807 15:10:35.863227 31545 heartbeater.cc:464] Registering TS with master...
I20260807 15:10:35.863781 31545 heartbeater.cc:511] Master 127.15.226.126:36387 requested a full tablet report, sending...
I20260807 15:10:35.864868 30828 ts_manager.cc:194] Registered new tserver with Master: df82f490d9c84676bdbe2a814347a678 (127.15.226.67:39137)
I20260807 15:10:35.865445 30828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.67@KRBTEST.COM'} at 127.15.226.67:49245
I20260807 15:10:35.873422 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:10:35.883620 30899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:35.875641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48694 (local address 127.15.226.126:36387)
0807 15:10:35.875787 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:35.875795 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:35.875986 (+ 191us) server_negotiation.cc:407] Connection header received
0807 15:10:35.876151 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:35.876161 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:35.876208 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:35.876317 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:10:35.877109 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.877664 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.878542 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.878830 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.878946 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:35.880426 (+ 1480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:10:35.880438 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:10:35.880440 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:10:35.880475 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:10:35.881724 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:35.882182 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:35.882185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:35.882187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:35.882276 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:10:35.882598 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:10:35.882601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:10:35.882603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:10:35.882898 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:10:35.883052 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:35.883306 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 15:10:35.883449 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 15:10:35.886904 30828 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:10:35.890000 30828 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:10:35.905619 31564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:35.900140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:42483 (local address 127.0.0.1:58166)
0807 15:10:35.900672 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:35.900688 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:35.900777 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:35.901521 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:35.901524 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:35.901538 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:35.901613 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:35.901622 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.902835 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.902841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:35.903899 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:35.903922 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.905080 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.905083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:35.905194 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:35.905284 (+ 90us) client_negotiation.cc:769] Sending connection context
0807 15:10:35.905340 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 15:10:35.905399 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":50,"thread_start_us":98,"threads_started":1}
I20260807 15:10:35.905707 31565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:35.900347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58166 (local address 127.15.226.65:42483)
0807 15:10:35.901039 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:35.901045 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:35.901070 (+ 25us) server_negotiation.cc:407] Connection header received
0807 15:10:35.901163 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:35.901170 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:35.901370 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:35.901764 (+ 394us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:35.901878 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.902708 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.904053 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.904924 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.905249 (+ 325us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:35.905407 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:35.905464 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 15:10:35.905565 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":95,"threads_started":1}
I20260807 15:10:35.905772 31563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:35.899490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35276 (local address 127.15.226.66:41949)
0807 15:10:35.899983 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:35.899997 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:35.900030 (+ 33us) server_negotiation.cc:407] Connection header received
0807 15:10:35.900223 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:35.900232 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:35.900453 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:35.900612 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:35.901166 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.902392 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.904244 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.904845 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.904987 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:35.905160 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:35.905495 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 15:10:35.905596 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":140,"threads_started":1}
I20260807 15:10:35.906134 31566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:35.900461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37292 (local address 127.15.226.67:39137)
0807 15:10:35.900951 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:35.900963 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:35.900987 (+ 24us) server_negotiation.cc:407] Connection header received
0807 15:10:35.901061 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:35.901069 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:35.901265 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:35.901429 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:35.901704 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.902822 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.904292 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.905071 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.905713 (+ 642us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:35.905869 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:35.905925 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 15:10:35.906015 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":168,"threads_started":1}
I20260807 15:10:35.905637 31561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:35.899502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:39137 (local address 127.0.0.1:37292)
0807 15:10:35.900192 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:35.900208 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:35.900307 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:35.901412 (+ 1105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:35.901418 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:35.901450 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:35.901574 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:35.901583 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.902981 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.902984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:35.904167 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:35.904179 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.905192 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.905196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:35.905310 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:35.905330 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 15:10:35.905384 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 15:10:35.905472 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"mutex_wait_us":22,"thread_start_us":123,"threads_started":1}
I20260807 15:10:35.906775 31560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:35.899339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:41949 (local address 127.0.0.1:35276)
0807 15:10:35.899815 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:35.899862 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:35.900001 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:35.900835 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:35.900841 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:35.900867 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:35.901022 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:35.901030 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.902548 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.902552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:35.904086 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:35.904098 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:35.904990 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:35.904994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:35.905103 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:35.905140 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 15:10:35.905261 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 15:10:35.905399 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":40,"thread_start_us":118,"threads_started":1}
I20260807 15:10:35.909278 31433 tablet_service.cc:1467] Processing CreateTablet for tablet f647b099feb94ddbad9c6c3e3a9b5d40 (DEFAULT_TABLE table=test-table [id=01fd71c7fffe43239f3bb3efd73c80f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:35.909289 31036 tablet_service.cc:1467] Processing CreateTablet for tablet f647b099feb94ddbad9c6c3e3a9b5d40 (DEFAULT_TABLE table=test-table [id=01fd71c7fffe43239f3bb3efd73c80f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:35.909387 31238 tablet_service.cc:1467] Processing CreateTablet for tablet f647b099feb94ddbad9c6c3e3a9b5d40 (DEFAULT_TABLE table=test-table [id=01fd71c7fffe43239f3bb3efd73c80f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:10:35.910336 31433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f647b099feb94ddbad9c6c3e3a9b5d40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:35.910349 31036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f647b099feb94ddbad9c6c3e3a9b5d40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:35.910348 31238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f647b099feb94ddbad9c6c3e3a9b5d40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:10:35.917158 31567 tablet_bootstrap.cc:492] T f647b099feb94ddbad9c6c3e3a9b5d40 P df82f490d9c84676bdbe2a814347a678: Bootstrap starting.
I20260807 15:10:35.918612 31569 tablet_bootstrap.cc:492] T f647b099feb94ddbad9c6c3e3a9b5d40 P ca7162ce47e640ba9a74c2c1b853e879: Bootstrap starting.
I20260807 15:10:35.919004 31571 tablet_bootstrap.cc:492] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1: Bootstrap starting.
I20260807 15:10:35.919682 31567 tablet_bootstrap.cc:654] T f647b099feb94ddbad9c6c3e3a9b5d40 P df82f490d9c84676bdbe2a814347a678: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:35.921043 31569 tablet_bootstrap.cc:654] T f647b099feb94ddbad9c6c3e3a9b5d40 P ca7162ce47e640ba9a74c2c1b853e879: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:35.921365 31567 log.cc:824] T f647b099feb94ddbad9c6c3e3a9b5d40 P df82f490d9c84676bdbe2a814347a678: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:35.921496 31571 tablet_bootstrap.cc:654] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1: Neither blocks nor log segments found. Creating new log.
I20260807 15:10:35.921957 31569 log.cc:824] T f647b099feb94ddbad9c6c3e3a9b5d40 P ca7162ce47e640ba9a74c2c1b853e879: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:35.922315 31571 log.cc:824] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1: Log is configured to *not* fsync() on all Append() calls
I20260807 15:10:35.923777 31567 tablet_bootstrap.cc:492] T f647b099feb94ddbad9c6c3e3a9b5d40 P df82f490d9c84676bdbe2a814347a678: No bootstrap required, opened a new log
I20260807 15:10:35.923938 31569 tablet_bootstrap.cc:492] T f647b099feb94ddbad9c6c3e3a9b5d40 P ca7162ce47e640ba9a74c2c1b853e879: No bootstrap required, opened a new log
I20260807 15:10:35.924005 31567 ts_tablet_manager.cc:1397] T f647b099feb94ddbad9c6c3e3a9b5d40 P df82f490d9c84676bdbe2a814347a678: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 15:10:35.924162 31569 ts_tablet_manager.cc:1397] T f647b099feb94ddbad9c6c3e3a9b5d40 P ca7162ce47e640ba9a74c2c1b853e879: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 15:10:35.924384 31571 tablet_bootstrap.cc:492] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1: No bootstrap required, opened a new log
I20260807 15:10:35.924579 31571 ts_tablet_manager.cc:1397] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 15:10:35.927832 31569 raft_consensus.cc:348] T f647b099feb94ddbad9c6c3e3a9b5d40 P ca7162ce47e640ba9a74c2c1b853e879 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2115efa6ea79429cb0f72e4855ad03e1" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 41949 } } peers { permanent_uuid: "ca7162ce47e640ba9a74c2c1b853e879" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42483 } } peers { permanent_uuid: "df82f490d9c84676bdbe2a814347a678" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 39137 } }
I20260807 15:10:35.928103 31569 raft_consensus.cc:374] T f647b099feb94ddbad9c6c3e3a9b5d40 P ca7162ce47e640ba9a74c2c1b853e879 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:35.928180 31569 raft_consensus.cc:729] T f647b099feb94ddbad9c6c3e3a9b5d40 P ca7162ce47e640ba9a74c2c1b853e879 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca7162ce47e640ba9a74c2c1b853e879, State: Initialized, Role: FOLLOWER
I20260807 15:10:35.928145 31567 raft_consensus.cc:348] T f647b099feb94ddbad9c6c3e3a9b5d40 P df82f490d9c84676bdbe2a814347a678 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2115efa6ea79429cb0f72e4855ad03e1" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 41949 } } peers { permanent_uuid: "ca7162ce47e640ba9a74c2c1b853e879" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42483 } } peers { permanent_uuid: "df82f490d9c84676bdbe2a814347a678" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 39137 } }
I20260807 15:10:35.928412 31567 raft_consensus.cc:374] T f647b099feb94ddbad9c6c3e3a9b5d40 P df82f490d9c84676bdbe2a814347a678 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:35.928522 31567 raft_consensus.cc:729] T f647b099feb94ddbad9c6c3e3a9b5d40 P df82f490d9c84676bdbe2a814347a678 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df82f490d9c84676bdbe2a814347a678, State: Initialized, Role: FOLLOWER
I20260807 15:10:35.928771 31569 consensus_queue.cc:260] T f647b099feb94ddbad9c6c3e3a9b5d40 P ca7162ce47e640ba9a74c2c1b853e879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2115efa6ea79429cb0f72e4855ad03e1" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 41949 } } peers { permanent_uuid: "ca7162ce47e640ba9a74c2c1b853e879" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42483 } } peers { permanent_uuid: "df82f490d9c84676bdbe2a814347a678" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 39137 } }
I20260807 15:10:35.928973 31567 consensus_queue.cc:260] T f647b099feb94ddbad9c6c3e3a9b5d40 P df82f490d9c84676bdbe2a814347a678 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2115efa6ea79429cb0f72e4855ad03e1" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 41949 } } peers { permanent_uuid: "ca7162ce47e640ba9a74c2c1b853e879" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42483 } } peers { permanent_uuid: "df82f490d9c84676bdbe2a814347a678" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 39137 } }
I20260807 15:10:35.929539 31139 heartbeater.cc:503] Master 127.15.226.126:36387 was elected leader, sending a full tablet report...
I20260807 15:10:35.929600 31571 raft_consensus.cc:348] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2115efa6ea79429cb0f72e4855ad03e1" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 41949 } } peers { permanent_uuid: "ca7162ce47e640ba9a74c2c1b853e879" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42483 } } peers { permanent_uuid: "df82f490d9c84676bdbe2a814347a678" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 39137 } }
I20260807 15:10:35.929836 31571 raft_consensus.cc:374] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:10:35.929879 31569 ts_tablet_manager.cc:1428] T f647b099feb94ddbad9c6c3e3a9b5d40 P ca7162ce47e640ba9a74c2c1b853e879: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 15:10:35.929886 31571 raft_consensus.cc:729] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2115efa6ea79429cb0f72e4855ad03e1, State: Initialized, Role: FOLLOWER
I20260807 15:10:35.929966 31567 ts_tablet_manager.cc:1428] T f647b099feb94ddbad9c6c3e3a9b5d40 P df82f490d9c84676bdbe2a814347a678: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 15:10:35.930366 31545 heartbeater.cc:503] Master 127.15.226.126:36387 was elected leader, sending a full tablet report...
I20260807 15:10:35.930294 31571 consensus_queue.cc:260] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2115efa6ea79429cb0f72e4855ad03e1" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 41949 } } peers { permanent_uuid: "ca7162ce47e640ba9a74c2c1b853e879" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42483 } } peers { permanent_uuid: "df82f490d9c84676bdbe2a814347a678" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 39137 } }
I20260807 15:10:35.930982 31338 heartbeater.cc:503] Master 127.15.226.126:36387 was elected leader, sending a full tablet report...
I20260807 15:10:35.932264 31571 ts_tablet_manager.cc:1428] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1: Time spent starting tablet: real 0.008s user 0.000s sys 0.006s
W20260807 15:10:36.079774 31339 tablet.cc:2369] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:10:36.088068 31579 raft_consensus.cc:482] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:10:36.088295 31579 raft_consensus.cc:504] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2115efa6ea79429cb0f72e4855ad03e1" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 41949 } } peers { permanent_uuid: "ca7162ce47e640ba9a74c2c1b853e879" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42483 } } peers { permanent_uuid: "df82f490d9c84676bdbe2a814347a678" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 39137 } }
I20260807 15:10:36.089563 31579 leader_election.cc:288] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca7162ce47e640ba9a74c2c1b853e879 (127.15.226.65:42483), df82f490d9c84676bdbe2a814347a678 (127.15.226.67:39137)
W20260807 15:10:36.093621 31140 tablet.cc:2369] T f647b099feb94ddbad9c6c3e3a9b5d40 P ca7162ce47e640ba9a74c2c1b853e879: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:10:36.093839 31566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.089977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:57743 (local address 127.15.226.67:39137)
0807 15:10:36.090119 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:36.090124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:36.090368 (+ 244us) server_negotiation.cc:407] Connection header received
0807 15:10:36.090534 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:36.090536 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:36.090580 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:36.090710 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:36.090994 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.091479 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.092598 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.093200 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.093331 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.093386 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:36.093604 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 15:10:36.093696 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 15:10:36.093915 31340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.089840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:42483 (local address 127.15.226.66:58675)
0807 15:10:36.090070 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:36.090088 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:36.090204 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:36.090502 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:36.090505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:36.090521 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:36.090702 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:36.090713 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.091463 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.091466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:36.092336 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:36.092346 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.093453 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.093456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:36.093600 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.093625 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 15:10:36.093702 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 15:10:36.093782 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141}
I20260807 15:10:36.094293 31565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.089987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:58675 (local address 127.15.226.65:42483)
0807 15:10:36.090118 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:36.090125 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:36.090143 (+ 18us) server_negotiation.cc:407] Connection header received
0807 15:10:36.090343 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:36.090346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:36.090393 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:36.090488 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:10:36.090828 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.091340 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.092492 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.093219 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.093332 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.093386 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:36.094094 (+ 708us) server_negotiation.cc:299] Negotiation successful
0807 15:10:36.094177 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 15:10:36.094313 31581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.089878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:39137 (local address 127.15.226.66:57743)
0807 15:10:36.090257 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:36.090268 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:10:36.090405 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:10:36.090709 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:10:36.090712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:10:36.090728 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:10:36.090861 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:36.090867 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.091597 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.091602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:36.092457 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:10:36.092469 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.093320 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.093324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:10:36.093424 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.093458 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 15:10:36.094025 (+ 567us) client_negotiation.cc:240] Negotiation successful
0807 15:10:36.094111 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"mutex_wait_us":1,"thread_start_us":93,"threads_started":1}
I20260807 15:10:36.095180 31070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f647b099feb94ddbad9c6c3e3a9b5d40" candidate_uuid: "2115efa6ea79429cb0f72e4855ad03e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca7162ce47e640ba9a74c2c1b853e879" is_pre_election: true
I20260807 15:10:36.095198 31468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f647b099feb94ddbad9c6c3e3a9b5d40" candidate_uuid: "2115efa6ea79429cb0f72e4855ad03e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df82f490d9c84676bdbe2a814347a678" is_pre_election: true
I20260807 15:10:36.095566 31070 raft_consensus.cc:2454] T f647b099feb94ddbad9c6c3e3a9b5d40 P ca7162ce47e640ba9a74c2c1b853e879 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2115efa6ea79429cb0f72e4855ad03e1 in term 0.
I20260807 15:10:36.095570 31468 raft_consensus.cc:2454] T f647b099feb94ddbad9c6c3e3a9b5d40 P df82f490d9c84676bdbe2a814347a678 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2115efa6ea79429cb0f72e4855ad03e1 in term 0.
I20260807 15:10:36.096174 31178 leader_election.cc:302] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2115efa6ea79429cb0f72e4855ad03e1, ca7162ce47e640ba9a74c2c1b853e879; no voters:
W20260807 15:10:36.096283 31547 tablet.cc:2369] T f647b099feb94ddbad9c6c3e3a9b5d40 P df82f490d9c84676bdbe2a814347a678: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:10:36.096524 31579 raft_consensus.cc:2781] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:10:36.096670 31579 raft_consensus.cc:482] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:10:36.096735 31579 raft_consensus.cc:3037] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:36.097877 31579 raft_consensus.cc:504] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2115efa6ea79429cb0f72e4855ad03e1" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 41949 } } peers { permanent_uuid: "ca7162ce47e640ba9a74c2c1b853e879" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42483 } } peers { permanent_uuid: "df82f490d9c84676bdbe2a814347a678" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 39137 } }
I20260807 15:10:36.098286 31579 leader_election.cc:288] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1 [CANDIDATE]: Term 1 election: Requested vote from peers ca7162ce47e640ba9a74c2c1b853e879 (127.15.226.65:42483), df82f490d9c84676bdbe2a814347a678 (127.15.226.67:39137)
I20260807 15:10:36.098795 31070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f647b099feb94ddbad9c6c3e3a9b5d40" candidate_uuid: "2115efa6ea79429cb0f72e4855ad03e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca7162ce47e640ba9a74c2c1b853e879"
I20260807 15:10:36.098973 31070 raft_consensus.cc:3037] T f647b099feb94ddbad9c6c3e3a9b5d40 P ca7162ce47e640ba9a74c2c1b853e879 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:36.099592 31468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f647b099feb94ddbad9c6c3e3a9b5d40" candidate_uuid: "2115efa6ea79429cb0f72e4855ad03e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df82f490d9c84676bdbe2a814347a678"
I20260807 15:10:36.099732 31468 raft_consensus.cc:3037] T f647b099feb94ddbad9c6c3e3a9b5d40 P df82f490d9c84676bdbe2a814347a678 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:10:36.100348 31070 raft_consensus.cc:2454] T f647b099feb94ddbad9c6c3e3a9b5d40 P ca7162ce47e640ba9a74c2c1b853e879 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2115efa6ea79429cb0f72e4855ad03e1 in term 1.
I20260807 15:10:36.100673 31178 leader_election.cc:302] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2115efa6ea79429cb0f72e4855ad03e1, ca7162ce47e640ba9a74c2c1b853e879; no voters:
I20260807 15:10:36.100853 31579 raft_consensus.cc:2781] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:10:36.101159 31579 raft_consensus.cc:686] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1 [term 1 LEADER]: Becoming Leader. State: Replica: 2115efa6ea79429cb0f72e4855ad03e1, State: Running, Role: LEADER
I20260807 15:10:36.101315 31468 raft_consensus.cc:2454] T f647b099feb94ddbad9c6c3e3a9b5d40 P df82f490d9c84676bdbe2a814347a678 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2115efa6ea79429cb0f72e4855ad03e1 in term 1.
I20260807 15:10:36.101625 31579 consensus_queue.cc:237] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2115efa6ea79429cb0f72e4855ad03e1" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 41949 } } peers { permanent_uuid: "ca7162ce47e640ba9a74c2c1b853e879" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42483 } } peers { permanent_uuid: "df82f490d9c84676bdbe2a814347a678" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 39137 } }
I20260807 15:10:36.106137 30826 catalog_manager.cc:5495] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2115efa6ea79429cb0f72e4855ad03e1 (127.15.226.66). New cstate: current_term: 1 leader_uuid: "2115efa6ea79429cb0f72e4855ad03e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2115efa6ea79429cb0f72e4855ad03e1" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 41949 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca7162ce47e640ba9a74c2c1b853e879" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df82f490d9c84676bdbe2a814347a678" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 39137 } health_report { overall_health: UNKNOWN } } }
I20260807 15:10:36.148902 31563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.145177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35288 (local address 127.15.226.66:41949)
0807 15:10:36.145325 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 15:10:36.145329 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:10:36.145352 (+ 23us) server_negotiation.cc:407] Connection header received
0807 15:10:36.145460 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:10:36.145462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:10:36.145522 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:10:36.145621 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:10:36.145862 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.146381 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.147393 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:10:36.147625 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:10:36.147708 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:10:36.148042 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:10:36.148506 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:10:36.148603 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 15:10:36.148691 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 15:10:36.148745 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:10:36.155694 31070 raft_consensus.cc:1261] T f647b099feb94ddbad9c6c3e3a9b5d40 P ca7162ce47e640ba9a74c2c1b853e879 [term 1 FOLLOWER]: Refusing update from remote peer 2115efa6ea79429cb0f72e4855ad03e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:36.155694 31468 raft_consensus.cc:1261] T f647b099feb94ddbad9c6c3e3a9b5d40 P df82f490d9c84676bdbe2a814347a678 [term 1 FOLLOWER]: Refusing update from remote peer 2115efa6ea79429cb0f72e4855ad03e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:10:36.156572 31582 consensus_queue.cc:1035] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df82f490d9c84676bdbe2a814347a678" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 39137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:36.156795 31579 consensus_queue.cc:1035] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca7162ce47e640ba9a74c2c1b853e879" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:10:36.165877 31586 mvcc.cc:205] Tried to move back new op lower bound from 7315928826484944896 to 7315928826277109760. Current Snapshot: MvccSnapshot[applied={T|T < 7315928826484944896}]
I20260807 15:10:36.166352 31587 mvcc.cc:205] Tried to move back new op lower bound from 7315928826484944896 to 7315928826277109760. Current Snapshot: MvccSnapshot[applied={T|T < 7315928826484944896}]
I20260807 15:10:36.176841 30828 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:36.177079 30828 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:10:36.179966 30828 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb7e03e0b93142fcaecc9b358e18ebea: Sending DeleteTablet for 3 replicas of tablet f647b099feb94ddbad9c6c3e3a9b5d40
I20260807 15:10:36.180910 31433 tablet_service.cc:1514] Processing DeleteTablet for tablet f647b099feb94ddbad9c6c3e3a9b5d40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:36 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:37292
I20260807 15:10:36.180910 31238 tablet_service.cc:1514] Processing DeleteTablet for tablet f647b099feb94ddbad9c6c3e3a9b5d40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:36 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:35276
I20260807 15:10:36.181195 31036 tablet_service.cc:1514] Processing DeleteTablet for tablet f647b099feb94ddbad9c6c3e3a9b5d40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:10:36 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:58166
I20260807 15:10:36.181756 31597 tablet_replica.cc:330] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1: stopping tablet replica
I20260807 15:10:36.181867 31601 tablet_replica.cc:330] T f647b099feb94ddbad9c6c3e3a9b5d40 P ca7162ce47e640ba9a74c2c1b853e879: stopping tablet replica
I20260807 15:10:36.182108 31597 raft_consensus.cc:2229] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:10:36.182190 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 30936
I20260807 15:10:36.182190 31601 raft_consensus.cc:2229] T f647b099feb94ddbad9c6c3e3a9b5d40 P ca7162ce47e640ba9a74c2c1b853e879 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:10:36.182530 31597 raft_consensus.cc:2258] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:36.182566 31601 raft_consensus.cc:2258] T f647b099feb94ddbad9c6c3e3a9b5d40 P ca7162ce47e640ba9a74c2c1b853e879 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:36.183007 31599 tablet_replica.cc:330] T f647b099feb94ddbad9c6c3e3a9b5d40 P df82f490d9c84676bdbe2a814347a678: stopping tablet replica
I20260807 15:10:36.183300 31599 raft_consensus.cc:2229] T f647b099feb94ddbad9c6c3e3a9b5d40 P df82f490d9c84676bdbe2a814347a678 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:10:36.183552 31599 raft_consensus.cc:2258] T f647b099feb94ddbad9c6c3e3a9b5d40 P df82f490d9c84676bdbe2a814347a678 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:10:36.186020 31599 ts_tablet_manager.cc:1905] T f647b099feb94ddbad9c6c3e3a9b5d40 P df82f490d9c84676bdbe2a814347a678: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:36.187029 31597 ts_tablet_manager.cc:1905] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:10:36.189064 31599 ts_tablet_manager.cc:1918] T f647b099feb94ddbad9c6c3e3a9b5d40 P df82f490d9c84676bdbe2a814347a678: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:36.189165 31599 log.cc:1196] T f647b099feb94ddbad9c6c3e3a9b5d40 P df82f490d9c84676bdbe2a814347a678: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/f647b099feb94ddbad9c6c3e3a9b5d40
I20260807 15:10:36.189535 31599 ts_tablet_manager.cc:1939] T f647b099feb94ddbad9c6c3e3a9b5d40 P df82f490d9c84676bdbe2a814347a678: Deleting consensus metadata
I20260807 15:10:36.191404 31597 ts_tablet_manager.cc:1918] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:10:36.191560 31597 log.cc:1196] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/f647b099feb94ddbad9c6c3e3a9b5d40
I20260807 15:10:36.191996 31597 ts_tablet_manager.cc:1939] T f647b099feb94ddbad9c6c3e3a9b5d40 P 2115efa6ea79429cb0f72e4855ad03e1: Deleting consensus metadata
I20260807 15:10:36.193440 30808 catalog_manager.cc:4841] TS 2115efa6ea79429cb0f72e4855ad03e1 (127.15.226.66:41949): tablet f647b099feb94ddbad9c6c3e3a9b5d40 (table test-table [id=01fd71c7fffe43239f3bb3efd73c80f3]) successfully deleted
I20260807 15:10:36.197824 30807 catalog_manager.cc:4841] TS df82f490d9c84676bdbe2a814347a678 (127.15.226.67:39137): tablet f647b099feb94ddbad9c6c3e3a9b5d40 (table test-table [id=01fd71c7fffe43239f3bb3efd73c80f3]) successfully deleted
W20260807 15:10:36.206449 30808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.226.65:42483 (error 108)
I20260807 15:10:36.207274 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 31151
I20260807 15:10:36.209942 31560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:10:36.207520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:42483 (local address 127.0.0.1:58176)
0807 15:10:36.209734 (+ 2214us) negotiation.cc:107] Waiting for socket to connect
0807 15:10:36.209824 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.65:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2109}
W20260807 15:10:36.210104 30808 catalog_manager.cc:4584] TS ca7162ce47e640ba9a74c2c1b853e879 (127.15.226.65:42483): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f647b099feb94ddbad9c6c3e3a9b5d40: Network error: Client connection negotiation failed: client connection to 127.15.226.65:42483: connect: Connection refused (error 111)
I20260807 15:10:36.218716 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 31343
I20260807 15:10:36.228971 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 30776
2026-08-07T15:10:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2814 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:10:36 dist-test-slave-3kk6 krb5kdc[31610](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:10:36 dist-test-slave-3kk6 krb5kdc[31610](info): set up 2 sockets
Aug 07 15:10:36 dist-test-slave-3kk6 krb5kdc[31610](info): commencing operation
krb5kdc: starting...
W20260807 15:10:37.480515 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:10:37 dist-test-slave-3kk6 krb5kdc[31610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:10:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:10:37Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.226.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.226.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.226.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/rangeradmin_127.15.226.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.226.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/rangeradmin_127.15.226.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.226.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.226.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.226.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/rangerlookup_127.15.226.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.226.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/rangerlookup_127.15.226.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.226.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.226.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.226.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/HTTP_127.15.226.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.226.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/HTTP_127.15.226.84@KRBTEST.COM.keytab.
I20260807 15:10:37.682781 16265 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskwNTJgD/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/postgres -l logfile start
2026-08-07 15:10:40.440 UTC [31663] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 15:10:40.440 UTC [31663] LOG: listening on IPv4 address "127.15.226.84", port 54853
2026-08-07 15:10:40.444 UTC [31663] LOG: listening on Unix socket "/tmp/.s.PGSQL.54853"
2026-08-07 15:10:40.468 UTC [31667] LOG: database system was shut down at 2026-08-07 15:10:38 UTC
2026-08-07 15:10:40.473 UTC [31663] LOG: database system is ready to accept connections
I20260807 15:10:41.623608 16265 mini_postgres.cc:96] Postgres bound to 54853
2026-08-07 15:10:41.632 UTC [31678] FATAL: database "slave" does not exist
127.15.226.84:54853 - accepting connections
I20260807 15:10:41.633782 16265 mini_ranger.cc:161] Starting Ranger...
I20260807 15:10:41.653345 16265 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 15:10:42.207348 16265 mini_ranger.cc:87] Created ranger Postgres database
I20260807 15:10:42.207448 16265 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ranger-admin
2026-08-07 15:10:42,582 [I] DB FLAVOR :POSTGRES
2026-08-07 15:10:42,582 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 15:10:42,582 [I] Checking connection..
2026-08-07 15:10:42,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:54853/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 15:10:42,878 [I] Checking connection passed.
2026-08-07 15:10:42,878 [I] --------- Verifying version history table ---------
2026-08-07 15:10:42,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:54853/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:10:43,170 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 15:10:43,170 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:54853/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:10:43,443 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 15:10:43,443 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 15:10:43,443 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:54853/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 15:10:43.704 UTC [31772] WARNING: there is no transaction in progress
2026-08-07 15:10:43,714 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 15:10:43,714 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:54853/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:10:43,971 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 15:10:43,972 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 15:10:43,972 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:54853/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 15:10:44,257 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:54853/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 15:10:44,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:54853/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-3kk6.c.gcp-upstream.internal','N') ;"
2026-08-07 15:10:44,780 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:54853/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 15:10:45,062 [I] Table x_portal_user does not exist in database ranger
2026-08-07 15:10:45,063 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:54853/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 15:10:45,337 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 15:10:45,337 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:54853/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 15:10:45,590 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:54853/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 15:10:45,857 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 15:10:45,858 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:54853/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 15:10:46.886 UTC [31936] WARNING: there is no transaction in progress
2026-08-07 15:10:46.905 UTC [31936] WARNING: there is no transaction in progress
2026-08-07 15:10:46.923 UTC [31936] WARNING: there is no transaction in progress
2026-08-07 15:10:46.941 UTC [31936] WARNING: there is no transaction in progress
2026-08-07 15:10:46.959 UTC [31936] WARNING: there is no transaction in progress
2026-08-07 15:10:46.982 UTC [31936] WARNING: there is no transaction in progress
2026-08-07 15:10:47.043 UTC [31936] WARNING: there is no transaction in progress
2026-08-07 15:10:47.052 UTC [31936] WARNING: there is no transaction in progress
2026-08-07 15:10:47.068 UTC [31936] WARNING: there is no transaction in progress
2026-08-07 15:10:47.079 UTC [31936] WARNING: there is no transaction in progress
2026-08-07 15:10:47.091 UTC [31936] WARNING: there is no transaction in progress
2026-08-07 15:10:47.106 UTC [31936] WARNING: there is no transaction in progress
2026-08-07 15:10:47.116 UTC [31936] WARNING: there is no transaction in progress
2026-08-07 15:10:47.123 UTC [31936] WARNING: there is no transaction in progress
2026-08-07 15:10:47.131 UTC [31936] WARNING: there is no transaction in progress
2026-08-07 15:10:47.138 UTC [31936] WARNING: there is no transaction in progress
2026-08-07 15:10:47,850 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 15:10:47,851 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:54853/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 15:10:48,129 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 15:10:48,130 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:54853/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 15:10:48,389 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 15:10:48,390 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:54853/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 15:10:48,667 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 15:10:48,668 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:54853/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 15:10:48,937 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:54853/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 15:10:49,190 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:54853/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-3kk6.c.gcp-upstream.internal';"
2026-08-07 15:10:49,462 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 15:10:49,463 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:54853/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 15:10:49,767 [I] DB_PATCHES have already been applied
I20260807 15:10:49.776995 16265 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ranger-admin:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskwNTJgD/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 15:10:49.777109 16265 mini_ranger.cc:193] Using host: 127.15.226.84
I20260807 15:10:49.782425 16265 mini_ranger.cc:227] Ranger admin URL: http://127.15.226.84:54691
Aug 07, 2026 3:10:50 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 3:10:50 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 3:10:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 3:10:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 3:10:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 3:10:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 3:10:50 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 15:10:51.112140 16265 mini_ranger.cc:160] Time spent starting Ranger: real 9.478s user 0.001s sys 0.015s
Aug 07, 2026 3:11:13 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 3:11:13 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 3:11:13 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 3:11:13 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 3:11:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 3:11:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 3:11:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 3:11:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 3:11:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 3:11:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 3:11:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 3:11:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 3:11:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 3:11:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 3:11:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 3:11:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 3:11:13 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 3:11:14 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 15:11:15.464016 16265 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.226.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.226.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.226.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/rangerkms_127.15.226.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.226.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/rangerkms_127.15.226.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.226.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.226.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.226.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/HTTP_127.15.226.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.226.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/HTTP_127.15.226.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 15:11:15 dist-test-slave-3kk6 krb5kdc[31610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115475, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 15:11:15.583819 16265 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 15:11:15.598037 16265 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 15:11:16.556614 16265 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 15:11:16.556722 16265 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ranger-kms
I20260807 15:11:16.556818 16265 mini_ranger_kms.cc:227] Using postgres at 127.15.226.84:54853
2026-08-07 15:11:17,017 [I] DB FLAVOR :POSTGRES
2026-08-07 15:11:17,017 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 15:11:17,018 [I] Checking connection
2026-08-07 15:11:17,018 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:54853/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 15:11:17,352 [I] connection success
2026-08-07 15:11:17,352 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 15:11:17,352 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 15:11:17,352 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:54853/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 15:11:17,687 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 15:11:17,687 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 15:11:17,687 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 15:11:17,687 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:54853/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 15:11:17,960 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 15:11:18.473308 16265 mini_ranger_kms.cc:310] Created kms service
I20260807 15:11:18.667914 16265 mini_ranger_kms.cc:326] Created kudu user
I20260807 15:11:18.717067 16265 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 15:11:18 dist-test-slave-3kk6 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115475, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.226.84@KRBTEST.COM
I20260807 15:11:19.123258 16265 mini_ranger_kms.cc:384] Added ranger policy
I20260807 15:11:19.123471 16265 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ranger-kms:/tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 15:11:19.123499 16265 mini_ranger_kms.cc:240] Using host: 127.15.226.84
I20260807 15:11:19.128505 16265 mini_ranger_kms.cc:276] Ranger KMS PID: 32207
I20260807 15:11:19.128669 16265 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.226.84:34719
Aug 07, 2026 3:11:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 3:11:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 3:11:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 3:11:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 3:11:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 3:11:20 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 15:11:21 dist-test-slave-3kk6 krb5kdc[31610](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115481, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.226.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:21.898447 16265 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.315s user 0.009s sys 0.430s
I20260807 15:11:21.898610 16265 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 15:11:21.898823 16265 mini_ranger_kms.cc:401] 127.15.226.84:34719/kms/v1/keys
Aug 07 15:11:22 dist-test-slave-3kk6 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115481, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.226.84@KRBTEST.COM for HTTP/127.15.226.84@KRBTEST.COM
Aug 07 15:11:22 dist-test-slave-3kk6 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115481, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.226.84@KRBTEST.COM for HTTP/127.15.226.84@KRBTEST.COM
Aug 07 15:11:25 dist-test-slave-3kk6 krb5kdc[31610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:11:25.438359 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:44347
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:44347
--ranger_config_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.226.84:34719/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:25.584053 32284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:25.584386 32284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:25.584430 32284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:25.588788 32284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:11:25.588867 32284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:25.588888 32284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:25.588905 32284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:11:25.588922 32284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:11:25.595010 32284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.226.84:34719/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:44347
--ranger_config_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:44347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:11:25.596333 32284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:25.597498 32284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:25.607841 32293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:25.608065 32284 server_base.cc:1034] running on GCE node
W20260807 15:11:25.608037 32290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:25.607889 32289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:25.609015 32284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:25.610271 32284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:25.611477 32284 hybrid_clock.cc:648] HybridClock initialized: now 1786115485611450 us; error 53 us; skew 500 ppm
Aug 07 15:11:25 dist-test-slave-3kk6 krb5kdc[31610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115485, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:25.615010 32284 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:11:25.616163 32284 webserver.cc:460] Webserver started at http://127.15.226.126:44459/ using document root <none> and password file <none>
I20260807 15:11:25.616825 32284 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:25.616911 32284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:25.617182 32284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:11:25 dist-test-slave-3kk6 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115485, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for HTTP/127.15.226.84@KRBTEST.COM
I20260807 15:11:25.671361 32284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "002cc6ade5ed4b8c8dd8995ba2044bb1"
format_stamp: "Formatted at 2026-08-07 15:11:25 on dist-test-slave-3kk6"
server_key: "b705d447cbb1a3c7fc521d60607618c0"
server_key_iv: "5e4a3a08740e9a92fd215afb558df252"
server_key_version: "kuduclusterkey@0"
I20260807 15:11:25.672135 32284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "002cc6ade5ed4b8c8dd8995ba2044bb1"
format_stamp: "Formatted at 2026-08-07 15:11:25 on dist-test-slave-3kk6"
server_key: "b705d447cbb1a3c7fc521d60607618c0"
server_key_iv: "5e4a3a08740e9a92fd215afb558df252"
server_key_version: "kuduclusterkey@0"
I20260807 15:11:25.676664 32284 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:11:25.692131 32302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:25.693630 32284 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:11:25.693786 32284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "002cc6ade5ed4b8c8dd8995ba2044bb1"
format_stamp: "Formatted at 2026-08-07 15:11:25 on dist-test-slave-3kk6"
server_key: "b705d447cbb1a3c7fc521d60607618c0"
server_key_iv: "5e4a3a08740e9a92fd215afb558df252"
server_key_version: "kuduclusterkey@0"
I20260807 15:11:25.693940 32284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:25.711748 32284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:25.715508 32284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:25.716106 32284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:25.728076 32284 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:44347
I20260807 15:11:25.728116 32373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:44347 every 8 connection(s)
I20260807 15:11:25.729270 32284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:11:25.732381 32374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:25.739018 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 32284
I20260807 15:11:25.739171 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:11:25.739482 16265 external_mini_cluster.cc:1428] Setting key 9d2ffe6de19b89edd678374a4a5c32ea
I20260807 15:11:25.739677 32374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 002cc6ade5ed4b8c8dd8995ba2044bb1: Bootstrap starting.
I20260807 15:11:25.742864 32374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 002cc6ade5ed4b8c8dd8995ba2044bb1: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:25.743729 32374 log.cc:824] T 00000000000000000000000000000000 P 002cc6ade5ed4b8c8dd8995ba2044bb1: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:25.746500 32374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 002cc6ade5ed4b8c8dd8995ba2044bb1: No bootstrap required, opened a new log
Aug 07 15:11:25 dist-test-slave-3kk6 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:11:25.751801 32374 raft_consensus.cc:348] T 00000000000000000000000000000000 P 002cc6ade5ed4b8c8dd8995ba2044bb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "002cc6ade5ed4b8c8dd8995ba2044bb1" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44347 } }
I20260807 15:11:25.752177 32374 raft_consensus.cc:374] T 00000000000000000000000000000000 P 002cc6ade5ed4b8c8dd8995ba2044bb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:25.752277 32374 raft_consensus.cc:729] T 00000000000000000000000000000000 P 002cc6ade5ed4b8c8dd8995ba2044bb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 002cc6ade5ed4b8c8dd8995ba2044bb1, State: Initialized, Role: FOLLOWER
I20260807 15:11:25.752952 32374 consensus_queue.cc:260] T 00000000000000000000000000000000 P 002cc6ade5ed4b8c8dd8995ba2044bb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "002cc6ade5ed4b8c8dd8995ba2044bb1" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44347 } }
I20260807 15:11:25.753153 32374 raft_consensus.cc:388] T 00000000000000000000000000000000 P 002cc6ade5ed4b8c8dd8995ba2044bb1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:11:25.753250 32374 raft_consensus.cc:482] T 00000000000000000000000000000000 P 002cc6ade5ed4b8c8dd8995ba2044bb1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:11:25.753361 32374 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 002cc6ade5ed4b8c8dd8995ba2044bb1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:25.754722 32374 raft_consensus.cc:504] T 00000000000000000000000000000000 P 002cc6ade5ed4b8c8dd8995ba2044bb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "002cc6ade5ed4b8c8dd8995ba2044bb1" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44347 } }
I20260807 15:11:25.755154 32374 leader_election.cc:302] T 00000000000000000000000000000000 P 002cc6ade5ed4b8c8dd8995ba2044bb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 002cc6ade5ed4b8c8dd8995ba2044bb1; no voters:
I20260807 15:11:25.755596 32374 leader_election.cc:288] T 00000000000000000000000000000000 P 002cc6ade5ed4b8c8dd8995ba2044bb1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:11:25.755808 32380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 002cc6ade5ed4b8c8dd8995ba2044bb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:25.756103 32380 raft_consensus.cc:686] T 00000000000000000000000000000000 P 002cc6ade5ed4b8c8dd8995ba2044bb1 [term 1 LEADER]: Becoming Leader. State: Replica: 002cc6ade5ed4b8c8dd8995ba2044bb1, State: Running, Role: LEADER
I20260807 15:11:25.756541 32380 consensus_queue.cc:237] T 00000000000000000000000000000000 P 002cc6ade5ed4b8c8dd8995ba2044bb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "002cc6ade5ed4b8c8dd8995ba2044bb1" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44347 } }
I20260807 15:11:25.757084 32374 sys_catalog.cc:564] T 00000000000000000000000000000000 P 002cc6ade5ed4b8c8dd8995ba2044bb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:11:25.757367 32378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:25.741093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39416 (local address 127.15.226.126:44347)
0807 15:11:25.741707 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:25.741729 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:25.741944 (+ 215us) server_negotiation.cc:407] Connection header received
0807 15:11:25.743016 (+ 1072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:25.743044 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:25.743532 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:25.743942 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:25.744996 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.746037 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.747134 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:25.747680 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:25.747883 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:25.751037 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:25.751045 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:25.751064 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:25.751106 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:25.753235 (+ 2129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:25.753870 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:25.753877 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:25.753886 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:25.754025 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:25.754354 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:25.754358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:25.754360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:25.754973 (+ 613us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:25.755204 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:25.755963 (+ 759us) server_negotiation.cc:299] Negotiation successful
0807 15:11:25.756318 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":180,"threads_started":1}
I20260807 15:11:25.757989 32380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 002cc6ade5ed4b8c8dd8995ba2044bb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "002cc6ade5ed4b8c8dd8995ba2044bb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "002cc6ade5ed4b8c8dd8995ba2044bb1" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44347 } } }
I20260807 15:11:25.758112 32380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 002cc6ade5ed4b8c8dd8995ba2044bb1 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:25.758343 32380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 002cc6ade5ed4b8c8dd8995ba2044bb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 002cc6ade5ed4b8c8dd8995ba2044bb1. Latest consensus state: current_term: 1 leader_uuid: "002cc6ade5ed4b8c8dd8995ba2044bb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "002cc6ade5ed4b8c8dd8995ba2044bb1" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44347 } } }
I20260807 15:11:25.758419 32380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 002cc6ade5ed4b8c8dd8995ba2044bb1 [sys.catalog]: This master's current role is: LEADER
I20260807 15:11:25.759737 32374 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 15:11:25.760535 32397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:11:25.771296 32397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:11:25.780226 32397 catalog_manager.cc:1347] Generated new cluster ID: 1251b0ea05364158915a20e2b56272d7
I20260807 15:11:25.780455 32397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:11:25.792543 32397 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:11:25.794165 32397 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:11:25.802386 32397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 002cc6ade5ed4b8c8dd8995ba2044bb1: Generated new TSK 0
I20260807 15:11:25.803464 32397 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.226.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
I20260807 15:11:26.683403 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.226.84:34719/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:44347
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:26.864348 32439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:26.864861 32439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:26.864969 32439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:26.871775 32439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:26.871873 32439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:26.871968 32439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:11:26.878562 32439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.226.84:34719/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:44347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:11:26.879837 32439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:26.881017 32439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:26.895272 32451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:26.895282 32454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:26.895541 32439 server_base.cc:1034] running on GCE node
W20260807 15:11:26.895292 32452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:26.896323 32439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:26.897338 32439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:26.898598 32439 hybrid_clock.cc:648] HybridClock initialized: now 1786115486898568 us; error 61 us; skew 500 ppm
Aug 07 15:11:26 dist-test-slave-3kk6 krb5kdc[31610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115486, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:26.903700 32439 init.cc:377] Logged in from keytab as kudu/127.15.226.65@KRBTEST.COM (short username kudu)
I20260807 15:11:26.904943 32439 webserver.cc:460] Webserver started at http://127.15.226.65:37845/ using document root <none> and password file <none>
I20260807 15:11:26.905617 32439 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:26.905691 32439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:26.905968 32439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:11:26 dist-test-slave-3kk6 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115486, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for HTTP/127.15.226.84@KRBTEST.COM
I20260807 15:11:26.924126 32439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "cb27ebeed2044d51877fa9d1d790668b"
format_stamp: "Formatted at 2026-08-07 15:11:26 on dist-test-slave-3kk6"
server_key: "fd166659e09f4249d11335651331e0f7"
server_key_iv: "15255830a26d313edb310659bf648a6f"
server_key_version: "kuduclusterkey@0"
I20260807 15:11:26.924883 32439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "cb27ebeed2044d51877fa9d1d790668b"
format_stamp: "Formatted at 2026-08-07 15:11:26 on dist-test-slave-3kk6"
server_key: "fd166659e09f4249d11335651331e0f7"
server_key_iv: "15255830a26d313edb310659bf648a6f"
server_key_version: "kuduclusterkey@0"
I20260807 15:11:26.929553 32439 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 15:11:26.946959 32462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:26.948734 32439 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 15:11:26.949023 32439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "cb27ebeed2044d51877fa9d1d790668b"
format_stamp: "Formatted at 2026-08-07 15:11:26 on dist-test-slave-3kk6"
server_key: "fd166659e09f4249d11335651331e0f7"
server_key_iv: "15255830a26d313edb310659bf648a6f"
server_key_version: "kuduclusterkey@0"
I20260807 15:11:26.949297 32439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:26.975718 32439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:26.980144 32439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:26.980476 32439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:26.981431 32439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:26.982693 32439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:26.982813 32439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:26.982955 32439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:26.983042 32439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:27.037616 32439 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:36593
I20260807 15:11:27.039126 32439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:11:27.039880 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 32439
I20260807 15:11:27.040020 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:11:27.040378 16265 external_mini_cluster.cc:1428] Setting key d73c4c73cab56863fb391f4f391bcadd
I20260807 15:11:27.043838 32631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:36593 every 8 connection(s)
Aug 07 15:11:27 dist-test-slave-3kk6 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115486, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:11:27.059567 32639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.043489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:59357 (local address 127.15.226.126:44347)
0807 15:11:27.043800 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:27.043805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:27.043824 (+ 19us) server_negotiation.cc:407] Connection header received
0807 15:11:27.044727 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:27.044735 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:27.044804 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:27.044876 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:27.046782 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.047619 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.049499 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.049893 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.049981 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.053929 (+ 3948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:27.053934 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:27.053937 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:27.053969 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:27.056256 (+ 2287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:27.057021 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:27.057025 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:27.057027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:27.057133 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:27.057696 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:27.057700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:27.057702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:27.058013 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:27.058106 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:27.059133 (+ 1027us) server_negotiation.cc:299] Negotiation successful
0807 15:11:27.059293 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":82,"threads_started":1}
I20260807 15:11:27.062109 32637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.042237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44347 (local address 127.15.226.65:59357)
0807 15:11:27.043025 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:27.043091 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:27.044426 (+ 1335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:27.045346 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:27.045356 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:27.045768 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:27.046539 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.046563 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.047816 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.047819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.049334 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.049346 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.050232 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.050236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.050484 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.051334 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:27.051356 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:27.053668 (+ 2312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:27.056430 (+ 2762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:27.056436 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:27.056457 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:27.056881 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:27.057293 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:27.057297 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:27.057300 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:27.057573 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:27.058181 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:27.058188 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:27.058732 (+ 544us) client_negotiation.cc:769] Sending connection context
0807 15:11:27.060322 (+ 1590us) client_negotiation.cc:240] Negotiation successful
0807 15:11:27.060795 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":183,"threads_started":1}
I20260807 15:11:27.064399 32633 heartbeater.cc:347] Connected to a master server at 127.15.226.126:44347
I20260807 15:11:27.064705 32633 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:27.065424 32633 heartbeater.cc:511] Master 127.15.226.126:44347 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.226.66@KRBTEST.COM; defaulting to no policy
I20260807 15:11:27.067749 32325 ts_manager.cc:194] Registered new tserver with Master: cb27ebeed2044d51877fa9d1d790668b (127.15.226.65:36593)
I20260807 15:11:27.069444 32325 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.65@KRBTEST.COM'} at 127.15.226.65:59357
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
I20260807 15:11:27.094540 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.226.84:34719/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:44347
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:11:27 dist-test-slave-3kk6 krb5kdc[31610](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115487, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 15:11:27.257929 32643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:27.258337 32643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:27.258453 32643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:27.263293 32643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:27.263434 32643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:27.263556 32643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:11:27.276520 32643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.226.84:34719/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:44347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:11:27.278469 32643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:27.280419 32643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:27.293684 32659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:27.293884 32656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:27.293718 32661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:27.295133 32643 server_base.cc:1034] running on GCE node
I20260807 15:11:27.295627 32643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:27.296324 32643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:27.297537 32643 hybrid_clock.cc:648] HybridClock initialized: now 1786115487297508 us; error 43 us; skew 500 ppm
Aug 07 15:11:27 dist-test-slave-3kk6 krb5kdc[31610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115487, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:27.302187 32643 init.cc:377] Logged in from keytab as kudu/127.15.226.66@KRBTEST.COM (short username kudu)
I20260807 15:11:27.303344 32643 webserver.cc:460] Webserver started at http://127.15.226.66:45353/ using document root <none> and password file <none>
I20260807 15:11:27.303968 32643 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:27.304067 32643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:27.304360 32643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:11:27 dist-test-slave-3kk6 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115487, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for HTTP/127.15.226.84@KRBTEST.COM
I20260807 15:11:27.321547 32643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "7fef283699a744bd9291a4674bb8e519"
format_stamp: "Formatted at 2026-08-07 15:11:27 on dist-test-slave-3kk6"
server_key: "a27052b6e0e4f4a281ba84e66592f99d"
server_key_iv: "04632a0b9de30b5745c9172725a7978e"
server_key_version: "kuduclusterkey@0"
I20260807 15:11:27.322423 32643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "7fef283699a744bd9291a4674bb8e519"
format_stamp: "Formatted at 2026-08-07 15:11:27 on dist-test-slave-3kk6"
server_key: "a27052b6e0e4f4a281ba84e66592f99d"
server_key_iv: "04632a0b9de30b5745c9172725a7978e"
server_key_version: "kuduclusterkey@0"
I20260807 15:11:27.330375 32643 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.000s sys 0.003s
I20260807 15:11:27.355022 32675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:27.357151 32643 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260807 15:11:27.357327 32643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "7fef283699a744bd9291a4674bb8e519"
format_stamp: "Formatted at 2026-08-07 15:11:27 on dist-test-slave-3kk6"
server_key: "a27052b6e0e4f4a281ba84e66592f99d"
server_key_iv: "04632a0b9de30b5745c9172725a7978e"
server_key_version: "kuduclusterkey@0"
I20260807 15:11:27.357491 32643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:27.395455 32643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:27.400056 32643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:27.400377 32643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:27.401202 32643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:27.402534 32643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:27.402715 32643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 15:11:27.402837 32643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:27.402905 32643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:27.427117 32643 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:43129
I20260807 15:11:27.428948 32643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:11:27.431907 376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:43129 every 8 connection(s)
I20260807 15:11:27.436897 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 32643
I20260807 15:11:27.437036 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:11:27.437352 16265 external_mini_cluster.cc:1428] Setting key 885a789ccacede88ab90aecc4fb8d3b7
WARNING: no policy specified for kudu/127.15.226.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.67@KRBTEST.COM" created.
Aug 07 15:11:27 dist-test-slave-3kk6 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115487, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:11:27.469388 32639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.435941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:41363 (local address 127.15.226.126:44347)
0807 15:11:27.436088 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:27.436093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:27.445045 (+ 8952us) server_negotiation.cc:407] Connection header received
0807 15:11:27.446855 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:27.446860 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:27.446922 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:27.446977 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:27.448984 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.449928 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.452163 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.452561 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.452667 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.462389 (+ 9722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:27.462394 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:27.462398 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:27.462440 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:27.465096 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:27.466228 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:27.466232 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:27.466236 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:27.466360 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:27.467583 (+ 1223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:27.467587 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:27.467589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:27.467979 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:27.468095 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:27.469009 (+ 914us) server_negotiation.cc:299] Negotiation successful
0807 15:11:27.469189 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 15:11:27.472059 379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.434586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44347 (local address 127.15.226.66:41363)
0807 15:11:27.444797 (+ 10211us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:27.444886 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:27.446495 (+ 1609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:27.447463 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:27.447473 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:27.447939 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:27.448756 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.448780 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.450134 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.450140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.451976 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.451993 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.452998 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.453004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.453284 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.454459 (+ 1175us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:27.454490 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:27.462039 (+ 7549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:27.465551 (+ 3512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:27.465560 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:27.465580 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:27.466069 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:27.466502 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:27.466505 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:27.466507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:27.467436 (+ 929us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:27.468191 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:27.468199 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:27.468648 (+ 449us) client_negotiation.cc:769] Sending connection context
0807 15:11:27.470054 (+ 1406us) client_negotiation.cc:240] Negotiation successful
0807 15:11:27.470488 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7998,"thread_start_us":233,"threads_started":1}
I20260807 15:11:27.474059 377 heartbeater.cc:347] Connected to a master server at 127.15.226.126:44347
I20260807 15:11:27.474347 377 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:27.475054 377 heartbeater.cc:511] Master 127.15.226.126:44347 requested a full tablet report, sending...
I20260807 15:11:27.476414 32325 ts_manager.cc:194] Registered new tserver with Master: 7fef283699a744bd9291a4674bb8e519 (127.15.226.66:43129)
I20260807 15:11:27.477223 32325 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.66@KRBTEST.COM'} at 127.15.226.66:41363
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
I20260807 15:11:27.487353 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.226.84:34719/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:44347
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:37591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:11:27.696520 383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:11:27.697014 383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:11:27.697116 383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:11:27.703292 383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:11:27.703456 383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:11:27.703575 383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:11:27.712160 383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:37591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.226.84:34719/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:44347
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:11:27.713948 383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:11:27.715545 383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:11:27.731032 392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:11:27.734581 390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:27.735211 383 server_base.cc:1034] running on GCE node
W20260807 15:11:27.744741 394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:11:27.745896 383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:11:27.747040 383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:11:27.754814 383 hybrid_clock.cc:648] HybridClock initialized: now 1786115487754716 us; error 94 us; skew 500 ppm
Aug 07 15:11:27 dist-test-slave-3kk6 krb5kdc[31610](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115487, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:11:27.760344 383 init.cc:377] Logged in from keytab as kudu/127.15.226.67@KRBTEST.COM (short username kudu)
Aug 07 15:11:27 dist-test-slave-3kk6 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115487, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for HTTP/127.15.226.84@KRBTEST.COM
I20260807 15:11:27.763084 383 webserver.cc:460] Webserver started at http://127.15.226.67:39459/ using document root <none> and password file <none>
I20260807 15:11:27.763877 383 fs_manager.cc:359] Metadata directory not provided
I20260807 15:11:27.764020 383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:11:27.764305 383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:11:27 dist-test-slave-3kk6 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115487, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for HTTP/127.15.226.84@KRBTEST.COM
I20260807 15:11:27.779917 383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "aa8578543c424d74a98372c17381304d"
format_stamp: "Formatted at 2026-08-07 15:11:27 on dist-test-slave-3kk6"
server_key: "2829229ad9f227c76831b0ce3f717f2a"
server_key_iv: "3ad305e8478fbba4f71db3961eef610a"
server_key_version: "kuduclusterkey@0"
I20260807 15:11:27.780823 383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "aa8578543c424d74a98372c17381304d"
format_stamp: "Formatted at 2026-08-07 15:11:27 on dist-test-slave-3kk6"
server_key: "2829229ad9f227c76831b0ce3f717f2a"
server_key_iv: "3ad305e8478fbba4f71db3961eef610a"
server_key_version: "kuduclusterkey@0"
I20260807 15:11:27.787019 383 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 15:11:27.805675 405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:27.807543 383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:11:27.807703 383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "aa8578543c424d74a98372c17381304d"
format_stamp: "Formatted at 2026-08-07 15:11:27 on dist-test-slave-3kk6"
server_key: "2829229ad9f227c76831b0ce3f717f2a"
server_key_iv: "3ad305e8478fbba4f71db3961eef610a"
server_key_version: "kuduclusterkey@0"
I20260807 15:11:27.807893 383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:11:27.829932 383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:11:27.833961 383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:11:27.834245 383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:11:27.835139 383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:11:27.836247 383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:11:27.836366 383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:27.836468 383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:11:27.836529 383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:11:27.881619 383 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:32975
I20260807 15:11:27.883165 383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:11:27.888607 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 383
I20260807 15:11:27.888756 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:11:27.889068 16265 external_mini_cluster.cc:1428] Setting key 020308b0f3d80ded421b9ae4155b5500
I20260807 15:11:27.887614 580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:32975 every 8 connection(s)
Aug 07 15:11:27 dist-test-slave-3kk6 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115487, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:11:27.939548 585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.897381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:44347 (local address 127.15.226.67:45031)
0807 15:11:27.911233 (+ 13852us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:27.911339 (+ 106us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:27.912974 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:27.913932 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:27.913943 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:27.914622 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:11:27.915764 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.915787 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.916921 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.916924 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.918364 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.918375 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.919173 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.919176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:27.919409 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.931093 (+ 11684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:11:27.931123 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:11:27.933188 (+ 2065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:11:27.935712 (+ 2524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:27.935720 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:27.935745 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:27.936159 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:27.936553 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:11:27.936557 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:11:27.936559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:11:27.936776 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:11:27.937324 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:11:27.937332 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:11:27.937691 (+ 359us) client_negotiation.cc:769] Sending connection context
0807 15:11:27.937911 (+ 220us) client_negotiation.cc:240] Negotiation successful
0807 15:11:27.938321 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13458,"thread_start_us":208,"threads_started":1}
I20260807 15:11:27.940368 32639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.898606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:45031 (local address 127.15.226.126:44347)
0807 15:11:27.898757 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:27.898762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:27.911509 (+ 12747us) server_negotiation.cc:407] Connection header received
0807 15:11:27.913312 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:27.913316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:27.913381 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:27.913436 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:27.915939 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.916758 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.918512 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.918877 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.918954 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.933506 (+ 14552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:27.933510 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:27.933513 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:27.933550 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:27.935537 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:27.936325 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:27.936330 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:27.936332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:27.936439 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:27.936896 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:27.936900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:27.936902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:27.937151 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:27.937258 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:27.940073 (+ 2815us) server_negotiation.cc:299] Negotiation successful
0807 15:11:27.940215 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 15:11:27.942543 581 heartbeater.cc:347] Connected to a master server at 127.15.226.126:44347
I20260807 15:11:27.942837 581 heartbeater.cc:464] Registering TS with master...
I20260807 15:11:27.943486 581 heartbeater.cc:511] Master 127.15.226.126:44347 requested a full tablet report, sending...
I20260807 15:11:27.944690 32325 ts_manager.cc:194] Registered new tserver with Master: aa8578543c424d74a98372c17381304d (127.15.226.67:32975)
I20260807 15:11:27.945477 32325 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.67@KRBTEST.COM'} at 127.15.226.67:45031
I20260807 15:11:27.955346 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:11:27.969058 32639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.958122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39426 (local address 127.15.226.126:44347)
0807 15:11:27.958262 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:27.958266 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:27.958865 (+ 599us) server_negotiation.cc:407] Connection header received
0807 15:11:27.959051 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:27.959055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:27.959102 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:27.959153 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:11:27.960344 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.961217 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.962612 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:27.962997 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:27.963078 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:27.965157 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:11:27.965161 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:11:27.965164 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:11:27.965198 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:11:27.967080 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:27.967623 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:27.967627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:27.967629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:27.967718 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:11:27.968041 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:11:27.968044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:11:27.968046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:11:27.968285 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:11:27.968387 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:27.968759 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 15:11:27.968887 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 15:11:27.973635 32325 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39426:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:11:27.978440 32325 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:11:28.009294 602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.989508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:43129 (local address 127.0.0.1:60180)
0807 15:11:27.994432 (+ 4924us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:27.994512 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:27.995402 (+ 890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:27.996206 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:27.996213 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:27.996237 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:27.996386 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:27.996399 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.002901 (+ 6502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.002905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:28.004497 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:28.004513 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.008610 (+ 4097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.008614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:28.008761 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:28.008828 (+ 67us) client_negotiation.cc:769] Sending connection context
0807 15:11:28.008959 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 15:11:28.009122 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4697,"thread_start_us":190,"threads_started":1}
I20260807 15:11:28.010314 599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.989652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60180 (local address 127.15.226.66:43129)
0807 15:11:27.995334 (+ 5682us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:27.995342 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:27.995368 (+ 26us) server_negotiation.cc:407] Connection header received
0807 15:11:27.995583 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:27.995589 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:27.995850 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:27.996013 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:27.996544 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.002739 (+ 6195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.004676 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.008204 (+ 3528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.008334 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:28.008532 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:28.010039 (+ 1507us) server_negotiation.cc:299] Negotiation successful
0807 15:11:28.010160 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5561,"thread_start_us":177,"threads_started":1}
I20260807 15:11:28.015239 598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.992372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:36593 (local address 127.0.0.1:33268)
0807 15:11:27.998987 (+ 6615us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:27.999002 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:27.999135 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:28.001336 (+ 2201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:28.001339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:28.001358 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:28.001530 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:28.001538 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.012098 (+ 10560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.012101 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:28.013283 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:28.013296 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.014729 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.014733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:28.014875 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:28.014902 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 15:11:28.014984 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 15:11:28.015067 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6553,"mutex_wait_us":27,"thread_start_us":178,"threads_started":1}
I20260807 15:11:28.016002 601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.991432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44108 (local address 127.15.226.67:32975)
0807 15:11:27.998697 (+ 7265us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:27.998721 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:27.998753 (+ 32us) server_negotiation.cc:407] Connection header received
0807 15:11:28.001699 (+ 2946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:28.001707 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:28.001967 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:28.002160 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:28.006236 (+ 4076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.007535 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.013193 (+ 5658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.014587 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.015384 (+ 797us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:28.015578 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:28.015641 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 15:11:28.015745 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7115,"thread_start_us":220,"threads_started":1}
I20260807 15:11:28.016037 603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.992935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33268 (local address 127.15.226.65:36593)
0807 15:11:27.999397 (+ 6462us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:27.999405 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:27.999429 (+ 24us) server_negotiation.cc:407] Connection header received
0807 15:11:27.999502 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:27.999507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:27.999741 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:27.999969 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:28.010770 (+ 10801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.011919 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.013459 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.014367 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.014484 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:28.014635 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:28.015795 (+ 1160us) server_negotiation.cc:299] Negotiation successful
0807 15:11:28.015904 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6348,"thread_start_us":77,"threads_started":1}
I20260807 15:11:28.016312 600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:27.990866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:32975 (local address 127.0.0.1:44108)
0807 15:11:27.994877 (+ 4011us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:27.994892 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:27.995009 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:28.002331 (+ 7322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:28.002334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:28.002354 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:28.002478 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:28.002485 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.010626 (+ 8141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.010630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:28.012942 (+ 2312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:28.012955 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.014803 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.014808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:28.015077 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:28.015108 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 15:11:28.015167 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 15:11:28.015245 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3931,"mutex_wait_us":3,"thread_start_us":185,"threads_started":1}
I20260807 15:11:28.017652 32743 tablet_service.cc:1467] Processing CreateTablet for tablet f4c21d8344524cc7b82d0cc66c3b75cc (DEFAULT_TABLE table=test-table [id=d766bdbb3b784bd6963b6c3ec60715d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:28.020896 484 tablet_service.cc:1467] Processing CreateTablet for tablet f4c21d8344524cc7b82d0cc66c3b75cc (DEFAULT_TABLE table=test-table [id=d766bdbb3b784bd6963b6c3ec60715d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:28.022118 484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4c21d8344524cc7b82d0cc66c3b75cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:28.021878 32527 tablet_service.cc:1467] Processing CreateTablet for tablet f4c21d8344524cc7b82d0cc66c3b75cc (DEFAULT_TABLE table=test-table [id=d766bdbb3b784bd6963b6c3ec60715d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:11:28.023094 32527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4c21d8344524cc7b82d0cc66c3b75cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:28.029022 32743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4c21d8344524cc7b82d0cc66c3b75cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:11:28.031754 604 tablet_bootstrap.cc:492] T f4c21d8344524cc7b82d0cc66c3b75cc P aa8578543c424d74a98372c17381304d: Bootstrap starting.
I20260807 15:11:28.034972 604 tablet_bootstrap.cc:654] T f4c21d8344524cc7b82d0cc66c3b75cc P aa8578543c424d74a98372c17381304d: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:28.036042 604 log.cc:824] T f4c21d8344524cc7b82d0cc66c3b75cc P aa8578543c424d74a98372c17381304d: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:28.038561 604 tablet_bootstrap.cc:492] T f4c21d8344524cc7b82d0cc66c3b75cc P aa8578543c424d74a98372c17381304d: No bootstrap required, opened a new log
I20260807 15:11:28.038808 604 ts_tablet_manager.cc:1397] T f4c21d8344524cc7b82d0cc66c3b75cc P aa8578543c424d74a98372c17381304d: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 15:11:28.044255 604 raft_consensus.cc:348] T f4c21d8344524cc7b82d0cc66c3b75cc P aa8578543c424d74a98372c17381304d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fef283699a744bd9291a4674bb8e519" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43129 } } peers { permanent_uuid: "aa8578543c424d74a98372c17381304d" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 32975 } } peers { permanent_uuid: "cb27ebeed2044d51877fa9d1d790668b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36593 } }
I20260807 15:11:28.044746 604 raft_consensus.cc:374] T f4c21d8344524cc7b82d0cc66c3b75cc P aa8578543c424d74a98372c17381304d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:28.044871 604 raft_consensus.cc:729] T f4c21d8344524cc7b82d0cc66c3b75cc P aa8578543c424d74a98372c17381304d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8578543c424d74a98372c17381304d, State: Initialized, Role: FOLLOWER
I20260807 15:11:28.045444 604 consensus_queue.cc:260] T f4c21d8344524cc7b82d0cc66c3b75cc P aa8578543c424d74a98372c17381304d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fef283699a744bd9291a4674bb8e519" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43129 } } peers { permanent_uuid: "aa8578543c424d74a98372c17381304d" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 32975 } } peers { permanent_uuid: "cb27ebeed2044d51877fa9d1d790668b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36593 } }
I20260807 15:11:28.046869 604 ts_tablet_manager.cc:1428] T f4c21d8344524cc7b82d0cc66c3b75cc P aa8578543c424d74a98372c17381304d: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260807 15:11:28.047333 581 heartbeater.cc:503] Master 127.15.226.126:44347 was elected leader, sending a full tablet report...
I20260807 15:11:28.048228 609 tablet_bootstrap.cc:492] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519: Bootstrap starting.
I20260807 15:11:28.050937 609 tablet_bootstrap.cc:654] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:28.051964 609 log.cc:824] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:28.054442 609 tablet_bootstrap.cc:492] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519: No bootstrap required, opened a new log
I20260807 15:11:28.054800 609 ts_tablet_manager.cc:1397] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260807 15:11:28.058944 613 tablet_bootstrap.cc:492] T f4c21d8344524cc7b82d0cc66c3b75cc P cb27ebeed2044d51877fa9d1d790668b: Bootstrap starting.
I20260807 15:11:28.060995 609 raft_consensus.cc:348] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fef283699a744bd9291a4674bb8e519" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43129 } } peers { permanent_uuid: "aa8578543c424d74a98372c17381304d" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 32975 } } peers { permanent_uuid: "cb27ebeed2044d51877fa9d1d790668b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36593 } }
I20260807 15:11:28.061394 609 raft_consensus.cc:374] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:28.061488 609 raft_consensus.cc:729] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fef283699a744bd9291a4674bb8e519, State: Initialized, Role: FOLLOWER
I20260807 15:11:28.062069 609 consensus_queue.cc:260] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fef283699a744bd9291a4674bb8e519" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43129 } } peers { permanent_uuid: "aa8578543c424d74a98372c17381304d" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 32975 } } peers { permanent_uuid: "cb27ebeed2044d51877fa9d1d790668b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36593 } }
I20260807 15:11:28.063413 609 ts_tablet_manager.cc:1428] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519: Time spent starting tablet: real 0.008s user 0.005s sys 0.002s
I20260807 15:11:28.063937 377 heartbeater.cc:503] Master 127.15.226.126:44347 was elected leader, sending a full tablet report...
I20260807 15:11:28.065058 613 tablet_bootstrap.cc:654] T f4c21d8344524cc7b82d0cc66c3b75cc P cb27ebeed2044d51877fa9d1d790668b: Neither blocks nor log segments found. Creating new log.
I20260807 15:11:28.066756 613 log.cc:824] T f4c21d8344524cc7b82d0cc66c3b75cc P cb27ebeed2044d51877fa9d1d790668b: Log is configured to *not* fsync() on all Append() calls
I20260807 15:11:28.070415 613 tablet_bootstrap.cc:492] T f4c21d8344524cc7b82d0cc66c3b75cc P cb27ebeed2044d51877fa9d1d790668b: No bootstrap required, opened a new log
I20260807 15:11:28.070704 613 ts_tablet_manager.cc:1397] T f4c21d8344524cc7b82d0cc66c3b75cc P cb27ebeed2044d51877fa9d1d790668b: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260807 15:11:28.072664 32633 heartbeater.cc:503] Master 127.15.226.126:44347 was elected leader, sending a full tablet report...
I20260807 15:11:28.076857 613 raft_consensus.cc:348] T f4c21d8344524cc7b82d0cc66c3b75cc P cb27ebeed2044d51877fa9d1d790668b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fef283699a744bd9291a4674bb8e519" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43129 } } peers { permanent_uuid: "aa8578543c424d74a98372c17381304d" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 32975 } } peers { permanent_uuid: "cb27ebeed2044d51877fa9d1d790668b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36593 } }
I20260807 15:11:28.077239 613 raft_consensus.cc:374] T f4c21d8344524cc7b82d0cc66c3b75cc P cb27ebeed2044d51877fa9d1d790668b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:11:28.077382 613 raft_consensus.cc:729] T f4c21d8344524cc7b82d0cc66c3b75cc P cb27ebeed2044d51877fa9d1d790668b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb27ebeed2044d51877fa9d1d790668b, State: Initialized, Role: FOLLOWER
I20260807 15:11:28.077957 613 consensus_queue.cc:260] T f4c21d8344524cc7b82d0cc66c3b75cc P cb27ebeed2044d51877fa9d1d790668b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fef283699a744bd9291a4674bb8e519" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43129 } } peers { permanent_uuid: "aa8578543c424d74a98372c17381304d" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 32975 } } peers { permanent_uuid: "cb27ebeed2044d51877fa9d1d790668b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36593 } }
I20260807 15:11:28.079380 613 ts_tablet_manager.cc:1428] T f4c21d8344524cc7b82d0cc66c3b75cc P cb27ebeed2044d51877fa9d1d790668b: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
W20260807 15:11:28.147058 582 tablet.cc:2369] T f4c21d8344524cc7b82d0cc66c3b75cc P aa8578543c424d74a98372c17381304d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 15:11:28 dist-test-slave-3kk6 krb5kdc[31610](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115487, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for HTTP/127.15.226.84@KRBTEST.COM
W20260807 15:11:28.193221 378 tablet.cc:2369] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:11:28.234015 614 raft_consensus.cc:482] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:11:28.234354 614 raft_consensus.cc:504] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fef283699a744bd9291a4674bb8e519" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43129 } } peers { permanent_uuid: "aa8578543c424d74a98372c17381304d" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 32975 } } peers { permanent_uuid: "cb27ebeed2044d51877fa9d1d790668b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36593 } }
I20260807 15:11:28.235723 614 leader_election.cc:288] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aa8578543c424d74a98372c17381304d (127.15.226.67:32975), cb27ebeed2044d51877fa9d1d790668b (127.15.226.65:36593)
I20260807 15:11:28.240885 603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:28.236259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:49417 (local address 127.15.226.65:36593)
0807 15:11:28.236403 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:28.236407 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:28.236774 (+ 367us) server_negotiation.cc:407] Connection header received
0807 15:11:28.236982 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:28.236985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:28.237040 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:28.237119 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:28.237856 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.238493 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.239740 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.240152 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.240305 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:28.240365 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:28.240648 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 15:11:28.240737 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 15:11:28.241436 621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:28.236005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:36593 (local address 127.15.226.66:49417)
0807 15:11:28.236670 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:28.236684 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:28.236843 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:28.237238 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:28.237240 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:28.237256 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:28.237659 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:28.237669 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.238620 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.238626 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:28.239591 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:28.239602 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.240305 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.240309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:28.240455 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:28.240505 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 15:11:28.241141 (+ 636us) client_negotiation.cc:240] Negotiation successful
0807 15:11:28.241230 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"mutex_wait_us":85,"thread_start_us":182,"threads_started":1}
I20260807 15:11:28.242287 32539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4c21d8344524cc7b82d0cc66c3b75cc" candidate_uuid: "7fef283699a744bd9291a4674bb8e519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb27ebeed2044d51877fa9d1d790668b" is_pre_election: true
I20260807 15:11:28.242714 32539 raft_consensus.cc:2454] T f4c21d8344524cc7b82d0cc66c3b75cc P cb27ebeed2044d51877fa9d1d790668b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fef283699a744bd9291a4674bb8e519 in term 0.
I20260807 15:11:28.243314 32677 leader_election.cc:302] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fef283699a744bd9291a4674bb8e519, cb27ebeed2044d51877fa9d1d790668b; no voters:
I20260807 15:11:28.243676 614 raft_consensus.cc:2781] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:11:28.243801 614 raft_consensus.cc:482] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:11:28.243878 614 raft_consensus.cc:3037] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:28.245751 614 raft_consensus.cc:504] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fef283699a744bd9291a4674bb8e519" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43129 } } peers { permanent_uuid: "aa8578543c424d74a98372c17381304d" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 32975 } } peers { permanent_uuid: "cb27ebeed2044d51877fa9d1d790668b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36593 } }
I20260807 15:11:28.246220 614 leader_election.cc:288] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519 [CANDIDATE]: Term 1 election: Requested vote from peers aa8578543c424d74a98372c17381304d (127.15.226.67:32975), cb27ebeed2044d51877fa9d1d790668b (127.15.226.65:36593)
I20260807 15:11:28.246717 32539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4c21d8344524cc7b82d0cc66c3b75cc" candidate_uuid: "7fef283699a744bd9291a4674bb8e519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb27ebeed2044d51877fa9d1d790668b"
I20260807 15:11:28.246912 601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:28.236171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:56809 (local address 127.15.226.67:32975)
0807 15:11:28.236328 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:28.236334 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:28.236695 (+ 361us) server_negotiation.cc:407] Connection header received
0807 15:11:28.236922 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:28.236927 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:28.236992 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:28.237075 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:11:28.237849 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.238489 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.243077 (+ 4588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.243762 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.243909 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:28.243967 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:28.246718 (+ 2751us) server_negotiation.cc:299] Negotiation successful
0807 15:11:28.246803 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 15:11:28.246929 623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:28.235925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:32975 (local address 127.15.226.66:56809)
0807 15:11:28.236521 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:28.236553 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 15:11:28.236750 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:11:28.237212 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:11:28.237215 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:11:28.237235 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:11:28.237653 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:28.237664 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.240397 (+ 2733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.240401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:28.241772 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:11:28.241790 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.246406 (+ 4616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.246411 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:11:28.246528 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:28.246567 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 15:11:28.246680 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 15:11:28.246766 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":55,"thread_start_us":238,"threads_started":1}
I20260807 15:11:28.246877 32539 raft_consensus.cc:3037] T f4c21d8344524cc7b82d0cc66c3b75cc P cb27ebeed2044d51877fa9d1d790668b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:28.248454 32539 raft_consensus.cc:2454] T f4c21d8344524cc7b82d0cc66c3b75cc P cb27ebeed2044d51877fa9d1d790668b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fef283699a744bd9291a4674bb8e519 in term 1.
I20260807 15:11:28.248893 32677 leader_election.cc:302] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fef283699a744bd9291a4674bb8e519, cb27ebeed2044d51877fa9d1d790668b; no voters:
I20260807 15:11:28.249245 514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4c21d8344524cc7b82d0cc66c3b75cc" candidate_uuid: "7fef283699a744bd9291a4674bb8e519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8578543c424d74a98372c17381304d"
I20260807 15:11:28.249239 515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4c21d8344524cc7b82d0cc66c3b75cc" candidate_uuid: "7fef283699a744bd9291a4674bb8e519" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa8578543c424d74a98372c17381304d" is_pre_election: true
I20260807 15:11:28.249528 514 raft_consensus.cc:3037] T f4c21d8344524cc7b82d0cc66c3b75cc P aa8578543c424d74a98372c17381304d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:11:28.250037 614 raft_consensus.cc:2781] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:11:28.250367 614 raft_consensus.cc:686] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519 [term 1 LEADER]: Becoming Leader. State: Replica: 7fef283699a744bd9291a4674bb8e519, State: Running, Role: LEADER
I20260807 15:11:28.250772 614 consensus_queue.cc:237] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fef283699a744bd9291a4674bb8e519" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43129 } } peers { permanent_uuid: "aa8578543c424d74a98372c17381304d" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 32975 } } peers { permanent_uuid: "cb27ebeed2044d51877fa9d1d790668b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36593 } }
I20260807 15:11:28.251097 514 raft_consensus.cc:2454] T f4c21d8344524cc7b82d0cc66c3b75cc P aa8578543c424d74a98372c17381304d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fef283699a744bd9291a4674bb8e519 in term 1.
I20260807 15:11:28.255335 32325 catalog_manager.cc:5495] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fef283699a744bd9291a4674bb8e519 (127.15.226.66). New cstate: current_term: 1 leader_uuid: "7fef283699a744bd9291a4674bb8e519" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fef283699a744bd9291a4674bb8e519" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa8578543c424d74a98372c17381304d" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 32975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb27ebeed2044d51877fa9d1d790668b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36593 } health_report { overall_health: UNKNOWN } } }
W20260807 15:11:28.291647 32635 tablet.cc:2369] T f4c21d8344524cc7b82d0cc66c3b75cc P cb27ebeed2044d51877fa9d1d790668b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:11:28.293526 599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:28.287793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43722 (local address 127.15.226.66:43129)
0807 15:11:28.287931 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 15:11:28.287935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:11:28.287951 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:11:28.288045 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:11:28.288048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:11:28.288123 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:11:28.288242 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:11:28.288711 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.289535 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.290841 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:11:28.291212 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:11:28.291333 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:11:28.291858 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:11:28.292492 (+ 634us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:11:28.292615 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 15:11:28.293310 (+ 695us) server_negotiation.cc:299] Negotiation successful
0807 15:11:28.293381 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 15:11:28.302291 514 raft_consensus.cc:1261] T f4c21d8344524cc7b82d0cc66c3b75cc P aa8578543c424d74a98372c17381304d [term 1 FOLLOWER]: Refusing update from remote peer 7fef283699a744bd9291a4674bb8e519: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:28.302361 32539 raft_consensus.cc:1261] T f4c21d8344524cc7b82d0cc66c3b75cc P cb27ebeed2044d51877fa9d1d790668b [term 1 FOLLOWER]: Refusing update from remote peer 7fef283699a744bd9291a4674bb8e519: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:11:28.303316 614 consensus_queue.cc:1035] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa8578543c424d74a98372c17381304d" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 32975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:28.303579 624 consensus_queue.cc:1035] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb27ebeed2044d51877fa9d1d790668b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:11:28.317919 627 mvcc.cc:205] Tried to move back new op lower bound from 7315929040075354112 to 7315929039880306688. Current Snapshot: MvccSnapshot[applied={T|T < 7315929040075354112}]
I20260807 15:11:28.321821 629 mvcc.cc:205] Tried to move back new op lower bound from 7315929040075354112 to 7315929039880306688. Current Snapshot: MvccSnapshot[applied={T|T < 7315929040075354112}]
I20260807 15:11:28.322322 628 mvcc.cc:205] Tried to move back new op lower bound from 7315929040075354112 to 7315929039880306688. Current Snapshot: MvccSnapshot[applied={T|T < 7315929040075354112}]
I20260807 15:11:28.339071 32324 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:28.339315 32324 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:11:28.342809 32324 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 002cc6ade5ed4b8c8dd8995ba2044bb1: Sending DeleteTablet for 3 replicas of tablet f4c21d8344524cc7b82d0cc66c3b75cc
I20260807 15:11:28.344710 32527 tablet_service.cc:1514] Processing DeleteTablet for tablet f4c21d8344524cc7b82d0cc66c3b75cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:28 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:33268
I20260807 15:11:28.345487 32743 tablet_service.cc:1514] Processing DeleteTablet for tablet f4c21d8344524cc7b82d0cc66c3b75cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:28 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:60180
I20260807 15:11:28.346079 642 tablet_replica.cc:330] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519: stopping tablet replica
I20260807 15:11:28.346109 484 tablet_service.cc:1514] Processing DeleteTablet for tablet f4c21d8344524cc7b82d0cc66c3b75cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:11:28 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:44108
I20260807 15:11:28.346449 642 raft_consensus.cc:2229] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:11:28.347254 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 32439
I20260807 15:11:28.347345 642 raft_consensus.cc:2258] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:28.351941 642 ts_tablet_manager.cc:1905] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:28.364014 642 ts_tablet_manager.cc:1918] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:11:28.364367 642 log.cc:1196] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/f4c21d8344524cc7b82d0cc66c3b75cc
I20260807 15:11:28.364889 642 ts_tablet_manager.cc:1939] T f4c21d8344524cc7b82d0cc66c3b75cc P 7fef283699a744bd9291a4674bb8e519: Deleting consensus metadata
I20260807 15:11:28.366729 32307 catalog_manager.cc:4841] TS 7fef283699a744bd9291a4674bb8e519 (127.15.226.66:43129): tablet f4c21d8344524cc7b82d0cc66c3b75cc (table test-table [id=d766bdbb3b784bd6963b6c3ec60715d9]) successfully deleted
I20260807 15:11:28.366791 643 tablet_replica.cc:330] T f4c21d8344524cc7b82d0cc66c3b75cc P aa8578543c424d74a98372c17381304d: stopping tablet replica
I20260807 15:11:28.367172 643 raft_consensus.cc:2229] T f4c21d8344524cc7b82d0cc66c3b75cc P aa8578543c424d74a98372c17381304d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:11:28.367625 643 raft_consensus.cc:2258] T f4c21d8344524cc7b82d0cc66c3b75cc P aa8578543c424d74a98372c17381304d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:11:28.369477 643 ts_tablet_manager.cc:1905] T f4c21d8344524cc7b82d0cc66c3b75cc P aa8578543c424d74a98372c17381304d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:11:28.372859 643 ts_tablet_manager.cc:1918] T f4c21d8344524cc7b82d0cc66c3b75cc P aa8578543c424d74a98372c17381304d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:11:28.373041 643 log.cc:1196] T f4c21d8344524cc7b82d0cc66c3b75cc P aa8578543c424d74a98372c17381304d: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/f4c21d8344524cc7b82d0cc66c3b75cc
I20260807 15:11:28.373453 643 ts_tablet_manager.cc:1939] T f4c21d8344524cc7b82d0cc66c3b75cc P aa8578543c424d74a98372c17381304d: Deleting consensus metadata
W20260807 15:11:28.373435 32304 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.226.65:36593 (error 108)
I20260807 15:11:28.375030 32303 catalog_manager.cc:4841] TS aa8578543c424d74a98372c17381304d (127.15.226.67:32975): tablet f4c21d8344524cc7b82d0cc66c3b75cc (table test-table [id=d766bdbb3b784bd6963b6c3ec60715d9]) successfully deleted
I20260807 15:11:28.375298 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 32643
I20260807 15:11:28.375499 600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:11:28.375003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:36593 (local address 127.0.0.1:40890)
0807 15:11:28.375305 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 15:11:28.375413 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.65:36593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212}
W20260807 15:11:28.387231 32304 catalog_manager.cc:4584] TS cb27ebeed2044d51877fa9d1d790668b (127.15.226.65:36593): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f4c21d8344524cc7b82d0cc66c3b75cc: Network error: Client connection negotiation failed: client connection to 127.15.226.65:36593: connect: Connection refused (error 111)
I20260807 15:11:28.394930 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 383
I20260807 15:11:28.413637 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 32284
I20260807 15:11:28.432160 16265 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 15:11:28.777949 16265 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 15:11:28.778103 16265 mini_ranger.cc:66] Stopping Ranger...
I20260807 15:11:29.124038 16265 mini_ranger.cc:68] Stopped Ranger
2026-08-07 15:11:29.124 UTC [31663] LOG: received smart shutdown request
2026-08-07 15:11:29.128 UTC [31663] LOG: background worker "logical replication launcher" (PID 31676) exited with exit code 1
2026-08-07 15:11:29.128 UTC [31668] LOG: shutting down
2026-08-07 15:11:29.303 UTC [31663] LOG: database system is shut down
2026-08-07T15:11:29Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53191 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:11:29 dist-test-slave-3kk6 krb5kdc[658](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:11:29 dist-test-slave-3kk6 krb5kdc[658](info): set up 2 sockets
Aug 07 15:11:29 dist-test-slave-3kk6 krb5kdc[658](info): commencing operation
krb5kdc: starting...
W20260807 15:11:31.012629 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.559s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:11:31 dist-test-slave-3kk6 krb5kdc[658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:11:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:11:31Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.226.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.226.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.226.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/rangeradmin_127.15.226.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.226.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/rangeradmin_127.15.226.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.226.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.226.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.226.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/rangerlookup_127.15.226.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.226.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/rangerlookup_127.15.226.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.226.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.226.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.226.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/HTTP_127.15.226.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.226.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/HTTP_127.15.226.84@KRBTEST.COM.keytab.
I20260807 15:11:31.214053 16265 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskwNTJgD/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/postgres -l logfile start
2026-08-07 15:11:33.987 UTC [704] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 15:11:33.987 UTC [704] LOG: listening on IPv4 address "127.15.226.84", port 50229
2026-08-07 15:11:33.991 UTC [704] LOG: listening on Unix socket "/tmp/.s.PGSQL.50229"
2026-08-07 15:11:34.015 UTC [708] LOG: database system was shut down at 2026-08-07 15:11:32 UTC
2026-08-07 15:11:34.019 UTC [704] LOG: database system is ready to accept connections
I20260807 15:11:35.200875 16265 mini_postgres.cc:96] Postgres bound to 50229
2026-08-07 15:11:35.209 UTC [724] FATAL: database "slave" does not exist
127.15.226.84:50229 - accepting connections
I20260807 15:11:35.210203 16265 mini_ranger.cc:161] Starting Ranger...
I20260807 15:11:35.230324 16265 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 15:11:35.805361 16265 mini_ranger.cc:87] Created ranger Postgres database
I20260807 15:11:35.805465 16265 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ranger-admin
2026-08-07 15:11:36,143 [I] DB FLAVOR :POSTGRES
2026-08-07 15:11:36,143 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 15:11:36,144 [I] Checking connection..
2026-08-07 15:11:36,144 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:50229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 15:11:36,441 [I] Checking connection passed.
2026-08-07 15:11:36,442 [I] --------- Verifying version history table ---------
2026-08-07 15:11:36,442 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:50229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:11:36,711 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 15:11:36,711 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:50229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:11:36,997 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 15:11:36,998 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 15:11:36,998 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:50229/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 15:11:37.279 UTC [811] WARNING: there is no transaction in progress
2026-08-07 15:11:37,287 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 15:11:37,288 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:50229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 15:11:37,539 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 15:11:37,540 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 15:11:37,540 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:50229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 15:11:37,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:50229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 15:11:38,069 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:50229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-3kk6.c.gcp-upstream.internal','N') ;"
2026-08-07 15:11:38,341 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:50229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 15:11:38,625 [I] Table x_portal_user does not exist in database ranger
2026-08-07 15:11:38,625 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:50229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 15:11:38,896 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 15:11:38,896 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:50229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 15:11:39,180 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:50229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 15:11:39,466 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 15:11:39,466 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:50229/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 15:11:40.569 UTC [968] WARNING: there is no transaction in progress
2026-08-07 15:11:40.587 UTC [968] WARNING: there is no transaction in progress
2026-08-07 15:11:40.606 UTC [968] WARNING: there is no transaction in progress
2026-08-07 15:11:40.623 UTC [968] WARNING: there is no transaction in progress
2026-08-07 15:11:40.641 UTC [968] WARNING: there is no transaction in progress
2026-08-07 15:11:40.661 UTC [968] WARNING: there is no transaction in progress
2026-08-07 15:11:40.724 UTC [968] WARNING: there is no transaction in progress
2026-08-07 15:11:40.732 UTC [968] WARNING: there is no transaction in progress
2026-08-07 15:11:40.748 UTC [968] WARNING: there is no transaction in progress
2026-08-07 15:11:40.760 UTC [968] WARNING: there is no transaction in progress
2026-08-07 15:11:40.773 UTC [968] WARNING: there is no transaction in progress
2026-08-07 15:11:40.789 UTC [968] WARNING: there is no transaction in progress
2026-08-07 15:11:40.800 UTC [968] WARNING: there is no transaction in progress
2026-08-07 15:11:40.807 UTC [968] WARNING: there is no transaction in progress
2026-08-07 15:11:40.815 UTC [968] WARNING: there is no transaction in progress
2026-08-07 15:11:40.823 UTC [968] WARNING: there is no transaction in progress
2026-08-07 15:11:41,609 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 15:11:41,610 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:50229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 15:11:41,892 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 15:11:41,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:50229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 15:11:42,186 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 15:11:42,187 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:50229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 15:11:42,479 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 15:11:42,480 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:50229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 15:11:42,746 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:50229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 15:11:43,016 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:50229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-3kk6.c.gcp-upstream.internal';"
2026-08-07 15:11:43,278 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 15:11:43,279 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:50229/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 15:11:43,541 [I] DB_PATCHES have already been applied
I20260807 15:11:43.548167 16265 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ranger-admin:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskwNTJgD/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 15:11:43.548239 16265 mini_ranger.cc:193] Using host: 127.15.226.84
I20260807 15:11:43.552824 16265 mini_ranger.cc:227] Ranger admin URL: http://127.15.226.84:38327
Aug 07, 2026 3:11:43 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 3:11:44 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 3:11:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 3:11:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 3:11:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 3:11:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 3:11:44 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 15:11:44.899583 16265 mini_ranger.cc:160] Time spent starting Ranger: real 9.689s user 0.005s sys 0.009s
Aug 07, 2026 3:12:05 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 3:12:05 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 07, 2026 3:12:05 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 3:12:05 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 3:12:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 3:12:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 3:12:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 3:12:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 3:12:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 3:12:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 3:12:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 3:12:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 3:12:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 3:12:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 3:12:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 3:12:05 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 3:12:05 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 3:12:06 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 15:12:07.443533 16265 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.226.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.226.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.226.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/rangerkms_127.15.226.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.226.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/rangerkms_127.15.226.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.226.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.226.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.226.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/HTTP_127.15.226.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.226.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/HTTP_127.15.226.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 15:12:07 dist-test-slave-3kk6 krb5kdc[658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115527, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 15:12:07.542615 16265 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 15:12:07.555102 16265 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 15:12:08.475503 16265 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 15:12:08.475589 16265 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ranger-kms
I20260807 15:12:08.475605 16265 mini_ranger_kms.cc:227] Using postgres at 127.15.226.84:50229
2026-08-07 15:12:08,707 [I] DB FLAVOR :POSTGRES
2026-08-07 15:12:08,707 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 15:12:08,707 [I] Checking connection
2026-08-07 15:12:08,707 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:50229/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 15:12:08,977 [I] connection success
2026-08-07 15:12:08,977 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 15:12:08,977 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 15:12:08,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:50229/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 15:12:09,231 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 15:12:09,231 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 15:12:09,231 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 15:12:09,231 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.226.84:50229/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 15:12:09,515 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 15:12:10.029034 16265 mini_ranger_kms.cc:310] Created kms service
I20260807 15:12:10.253034 16265 mini_ranger_kms.cc:326] Created kudu user
I20260807 15:12:10.302098 16265 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 15:12:10 dist-test-slave-3kk6 krb5kdc[658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115527, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.226.84@KRBTEST.COM
I20260807 15:12:10.769913 16265 mini_ranger_kms.cc:384] Added ranger policy
I20260807 15:12:10.770092 16265 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ranger-kms:/tmp/dist-test-taskwNTJgD/build/debug/bin/postgresql.jar:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskwNTJgD/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 15:12:10.770113 16265 mini_ranger_kms.cc:240] Using host: 127.15.226.84
I20260807 15:12:10.775157 16265 mini_ranger_kms.cc:276] Ranger KMS PID: 1247
I20260807 15:12:10.775262 16265 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.226.84:32929
Aug 07, 2026 3:12:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 3:12:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 3:12:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 3:12:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 3:12:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 3:12:12 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 15:12:13 dist-test-slave-3kk6 krb5kdc[658](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115533, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.226.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:13.495807 16265 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.953s user 0.018s sys 0.195s
I20260807 15:12:13.495978 16265 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 15:12:13.496065 16265 mini_ranger_kms.cc:401] 127.15.226.84:32929/kms/v1/keys
Aug 07 15:12:13 dist-test-slave-3kk6 krb5kdc[658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115533, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.226.84@KRBTEST.COM for HTTP/127.15.226.84@KRBTEST.COM
Aug 07 15:12:13 dist-test-slave-3kk6 krb5kdc[658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115533, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.226.84@KRBTEST.COM for HTTP/127.15.226.84@KRBTEST.COM
Aug 07 15:12:16 dist-test-slave-3kk6 krb5kdc[658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:12:16.470119 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:34813
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:34813
--ranger_config_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.226.84:32929/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:16.590421 1328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:16.590754 1328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:16.590799 1328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:16.595115 1328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:12:16.595193 1328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:16.595213 1328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:16.595271 1328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:12:16.595299 1328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:12:16.601143 1328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.226.84:32929/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:34813
--ranger_config_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:34813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:16.602329 1328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:16.603473 1328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:12:16.612977 1328 server_base.cc:1034] running on GCE node
W20260807 15:12:16.612921 1333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:16.612948 1341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:16.612921 1335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:16.613945 1328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:16.615263 1328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:16.616504 1328 hybrid_clock.cc:648] HybridClock initialized: now 1786115536616471 us; error 49 us; skew 500 ppm
Aug 07 15:12:16 dist-test-slave-3kk6 krb5kdc[658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115536, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:16.619676 1328 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:12:16.620652 1328 webserver.cc:460] Webserver started at http://127.15.226.126:40065/ using document root <none> and password file <none>
I20260807 15:12:16.621239 1328 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:16.621333 1328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:16.621590 1328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:12:16 dist-test-slave-3kk6 krb5kdc[658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115536, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for HTTP/127.15.226.84@KRBTEST.COM
I20260807 15:12:16.671080 1328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "6e4b306b689241a3a35a11259cff3810"
format_stamp: "Formatted at 2026-08-07 15:12:16 on dist-test-slave-3kk6"
server_key: "f0f0de41b69554b5fbb7f748e8de6e82"
server_key_iv: "661fe8162628421d30091d0060fcf644"
server_key_version: "kuduclusterkey@0"
I20260807 15:12:16.671779 1328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "6e4b306b689241a3a35a11259cff3810"
format_stamp: "Formatted at 2026-08-07 15:12:16 on dist-test-slave-3kk6"
server_key: "f0f0de41b69554b5fbb7f748e8de6e82"
server_key_iv: "661fe8162628421d30091d0060fcf644"
server_key_version: "kuduclusterkey@0"
I20260807 15:12:16.675944 1328 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 15:12:16.691360 1352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:16.692854 1328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:12:16.693023 1328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "6e4b306b689241a3a35a11259cff3810"
format_stamp: "Formatted at 2026-08-07 15:12:16 on dist-test-slave-3kk6"
server_key: "f0f0de41b69554b5fbb7f748e8de6e82"
server_key_iv: "661fe8162628421d30091d0060fcf644"
server_key_version: "kuduclusterkey@0"
I20260807 15:12:16.693194 1328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:16.725102 1328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:16.728704 1328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:16.729002 1328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:16.737866 1328 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:34813
I20260807 15:12:16.737866 1423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:34813 every 8 connection(s)
I20260807 15:12:16.739122 1328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:12:16.740760 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 1328
I20260807 15:12:16.740892 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:12:16.741204 16265 external_mini_cluster.cc:1428] Setting key dadaf46b9cbf7e9fd19ddd62c2f444a8
I20260807 15:12:16.742511 1424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:16.749680 1424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810: Bootstrap starting.
Aug 07 15:12:16 dist-test-slave-3kk6 krb5kdc[658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:12:16.752609 1424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:16.753543 1424 log.cc:824] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:16.755934 1424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810: No bootstrap required, opened a new log
I20260807 15:12:16.759740 1429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:16.742852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47666 (local address 127.15.226.126:34813)
0807 15:12:16.743331 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:16.743353 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:16.743398 (+ 45us) server_negotiation.cc:407] Connection header received
0807 15:12:16.744288 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:16.744312 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:16.744730 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:16.745120 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:16.746050 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:16.747558 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:16.748756 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:16.749452 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:16.749621 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:16.753103 (+ 3482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:16.753111 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:16.753131 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:16.753169 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:16.756085 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:16.756694 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:16.756701 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:16.756708 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:16.756852 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:16.757197 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:16.757200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:16.757202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:16.757745 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:16.758226 (+ 481us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:16.758397 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 15:12:16.758738 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":144,"threads_started":1}
I20260807 15:12:16.759799 1424 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4b306b689241a3a35a11259cff3810" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34813 } }
I20260807 15:12:16.760162 1424 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:16.760229 1424 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e4b306b689241a3a35a11259cff3810, State: Initialized, Role: FOLLOWER
I20260807 15:12:16.760694 1424 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4b306b689241a3a35a11259cff3810" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34813 } }
I20260807 15:12:16.760860 1424 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:12:16.760946 1424 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:12:16.761053 1424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:16.762113 1424 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4b306b689241a3a35a11259cff3810" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34813 } }
I20260807 15:12:16.762562 1424 leader_election.cc:302] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e4b306b689241a3a35a11259cff3810; no voters:
I20260807 15:12:16.762938 1424 leader_election.cc:288] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:12:16.763072 1431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:12:16.763293 1431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [term 1 LEADER]: Becoming Leader. State: Replica: 6e4b306b689241a3a35a11259cff3810, State: Running, Role: LEADER
I20260807 15:12:16.763688 1431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4b306b689241a3a35a11259cff3810" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34813 } }
I20260807 15:12:16.764271 1424 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:12:16.764935 1431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e4b306b689241a3a35a11259cff3810. Latest consensus state: current_term: 1 leader_uuid: "6e4b306b689241a3a35a11259cff3810" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4b306b689241a3a35a11259cff3810" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34813 } } }
I20260807 15:12:16.765050 1431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:16.766314 1432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e4b306b689241a3a35a11259cff3810" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4b306b689241a3a35a11259cff3810" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34813 } } }
I20260807 15:12:16.766495 1432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:16.766969 1424 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 15:12:16.767457 1439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:12:16.774294 1439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:12:16.784288 1439 catalog_manager.cc:1347] Generated new cluster ID: 2003786bb89d4f1c88885659ec555a15
I20260807 15:12:16.784508 1439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:12:16.795503 1439 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:12:16.796813 1439 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:12:16.805977 1439 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810: Generated new TSK 0
I20260807 15:12:16.807096 1439 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.226.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
I20260807 15:12:17.553884 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.226.84:32929/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34813
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:17.723181 1491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:17.723730 1491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:17.723868 1491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:17.731367 1491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:17.731565 1491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:17.731765 1491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:12:17.742462 1491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.226.84:32929/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:17.744331 1491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:17.745926 1491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:17.759791 1499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:17.760879 1502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:17.760919 1500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:17.761179 1491 server_base.cc:1034] running on GCE node
I20260807 15:12:17.761888 1491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:17.762856 1491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:17.764242 1491 hybrid_clock.cc:648] HybridClock initialized: now 1786115537764156 us; error 94 us; skew 500 ppm
Aug 07 15:12:17 dist-test-slave-3kk6 krb5kdc[658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115537, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:17.768638 1491 init.cc:377] Logged in from keytab as kudu/127.15.226.65@KRBTEST.COM (short username kudu)
I20260807 15:12:17.769932 1491 webserver.cc:460] Webserver started at http://127.15.226.65:35715/ using document root <none> and password file <none>
I20260807 15:12:17.770799 1491 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:17.770955 1491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:17.771277 1491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:12:17 dist-test-slave-3kk6 krb5kdc[658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115537, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for HTTP/127.15.226.84@KRBTEST.COM
I20260807 15:12:17.792646 1491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "7f98b732e2af4a05b09ee79a75cff680"
format_stamp: "Formatted at 2026-08-07 15:12:17 on dist-test-slave-3kk6"
server_key: "a7193fb39b7997ab360e78657b15e3e6"
server_key_iv: "30f28b02f340e6dcf58a5972c5e9f9d9"
server_key_version: "kuduclusterkey@0"
I20260807 15:12:17.793475 1491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "7f98b732e2af4a05b09ee79a75cff680"
format_stamp: "Formatted at 2026-08-07 15:12:17 on dist-test-slave-3kk6"
server_key: "a7193fb39b7997ab360e78657b15e3e6"
server_key_iv: "30f28b02f340e6dcf58a5972c5e9f9d9"
server_key_version: "kuduclusterkey@0"
I20260807 15:12:17.800201 1491 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260807 15:12:17.825881 1512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:17.831027 1491 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.003s sys 0.001s
I20260807 15:12:17.831218 1491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "7f98b732e2af4a05b09ee79a75cff680"
format_stamp: "Formatted at 2026-08-07 15:12:17 on dist-test-slave-3kk6"
server_key: "a7193fb39b7997ab360e78657b15e3e6"
server_key_iv: "30f28b02f340e6dcf58a5972c5e9f9d9"
server_key_version: "kuduclusterkey@0"
I20260807 15:12:17.831426 1491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:17.863648 1491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:17.867528 1491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:17.867767 1491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:17.868506 1491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:17.869493 1491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:17.869565 1491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:17.869661 1491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:17.869696 1491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:17.881701 1491 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:42331
I20260807 15:12:17.881718 1686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:42331 every 8 connection(s)
I20260807 15:12:17.882970 1491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:12:17.890528 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 1491
I20260807 15:12:17.890720 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:12:17.891039 16265 external_mini_cluster.cc:1428] Setting key 8d331599b153bd811c24524f513fc9cc
Aug 07 15:12:17 dist-test-slave-3kk6 krb5kdc[658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115537, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:12:17.902855 1691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:17.885537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34813 (local address 127.15.226.65:58077)
0807 15:12:17.886016 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:17.886073 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:17.887106 (+ 1033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:17.887706 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:17.887713 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:17.888001 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:17.888526 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:17.888546 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:17.889391 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:17.889394 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:17.890405 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:17.890418 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:17.892414 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:17.892422 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:17.892651 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:17.893453 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:17.893479 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:17.895385 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:17.897881 (+ 2496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:17.897888 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:17.897910 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:17.898269 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:17.898624 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:17.898627 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:17.898629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:17.898893 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:17.899467 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:17.899475 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:17.899854 (+ 379us) client_negotiation.cc:769] Sending connection context
0807 15:12:17.900083 (+ 229us) client_negotiation.cc:240] Negotiation successful
0807 15:12:17.900503 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":147,"threads_started":1}
I20260807 15:12:17.904536 1690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:17.885048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:58077 (local address 127.15.226.126:34813)
0807 15:12:17.885317 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:17.885321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:17.886196 (+ 875us) server_negotiation.cc:407] Connection header received
0807 15:12:17.887336 (+ 1140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:17.887340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:17.887405 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:17.887524 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:17.888686 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:17.889254 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:17.891480 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:17.891837 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:17.891942 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:17.895591 (+ 3649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:17.895598 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:17.895603 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:17.895642 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:17.897745 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:17.898407 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:17.898411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:17.898413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:17.898520 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:17.899022 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:17.899025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:17.899027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:17.899290 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:17.904039 (+ 4749us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:17.904153 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 15:12:17.904343 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":82,"threads_started":1}
I20260807 15:12:17.906076 1687 heartbeater.cc:347] Connected to a master server at 127.15.226.126:34813
WARNING: no policy specified for kudu/127.15.226.66@KRBTEST.COM; defaulting to no policy
I20260807 15:12:17.906376 1687 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:17.907095 1687 heartbeater.cc:511] Master 127.15.226.126:34813 requested a full tablet report, sending...
I20260807 15:12:17.909083 1374 ts_manager.cc:194] Registered new tserver with Master: 7f98b732e2af4a05b09ee79a75cff680 (127.15.226.65:42331)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.66@KRBTEST.COM" created.
I20260807 15:12:17.910921 1374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.65@KRBTEST.COM'} at 127.15.226.65:58077
Aug 07 15:12:17 dist-test-slave-3kk6 krb5kdc[658](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115537, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
I20260807 15:12:17.934448 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.226.84:32929/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34813
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:18.106014 1694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:18.106551 1694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:18.106743 1694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:18.113277 1694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:18.113391 1694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:18.113483 1694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:12:18.122009 1694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.226.84:32929/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:18.123601 1694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:18.125090 1694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:18.141052 1701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:18.141474 1704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:18.142882 1702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:18.143739 1694 server_base.cc:1034] running on GCE node
I20260807 15:12:18.144399 1694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:18.145406 1694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:18.146708 1694 hybrid_clock.cc:648] HybridClock initialized: now 1786115538146633 us; error 98 us; skew 500 ppm
Aug 07 15:12:18 dist-test-slave-3kk6 krb5kdc[658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115538, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:18.150990 1694 init.cc:377] Logged in from keytab as kudu/127.15.226.66@KRBTEST.COM (short username kudu)
I20260807 15:12:18.152390 1694 webserver.cc:460] Webserver started at http://127.15.226.66:36095/ using document root <none> and password file <none>
I20260807 15:12:18.153234 1694 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:18.153354 1694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:18.153735 1694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:12:18 dist-test-slave-3kk6 krb5kdc[658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115538, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for HTTP/127.15.226.84@KRBTEST.COM
I20260807 15:12:18.173712 1694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "084f8dc16f124199aba444b8de8c0f33"
format_stamp: "Formatted at 2026-08-07 15:12:18 on dist-test-slave-3kk6"
server_key: "5bd980c1803bb79f8c55374441a76ae7"
server_key_iv: "47719a0f09b669ac862ca4c235861c25"
server_key_version: "kuduclusterkey@0"
I20260807 15:12:18.174702 1694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "084f8dc16f124199aba444b8de8c0f33"
format_stamp: "Formatted at 2026-08-07 15:12:18 on dist-test-slave-3kk6"
server_key: "5bd980c1803bb79f8c55374441a76ae7"
server_key_iv: "47719a0f09b669ac862ca4c235861c25"
server_key_version: "kuduclusterkey@0"
I20260807 15:12:18.180725 1694 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.003s
I20260807 15:12:18.196735 1711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:18.198602 1694 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:12:18.198843 1694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "084f8dc16f124199aba444b8de8c0f33"
format_stamp: "Formatted at 2026-08-07 15:12:18 on dist-test-slave-3kk6"
server_key: "5bd980c1803bb79f8c55374441a76ae7"
server_key_iv: "47719a0f09b669ac862ca4c235861c25"
server_key_version: "kuduclusterkey@0"
I20260807 15:12:18.199005 1694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:18.223496 1694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:18.227638 1694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:18.227903 1694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:18.228704 1694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:18.229797 1694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:18.229872 1694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:18.229970 1694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:18.230018 1694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:18.242954 1694 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:43217
I20260807 15:12:18.244390 1694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:12:18.245478 1877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:43217 every 8 connection(s)
I20260807 15:12:18.251789 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 1694
I20260807 15:12:18.251938 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:12:18.252290 16265 external_mini_cluster.cc:1428] Setting key 71f3aaebaa119db5a67f1d6e6b8d40cd
Aug 07 15:12:18 dist-test-slave-3kk6 krb5kdc[658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115538, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:12:18.264235 1690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:18.247662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:42857 (local address 127.15.226.126:34813)
0807 15:12:18.247855 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:18.247860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:18.249281 (+ 1421us) server_negotiation.cc:407] Connection header received
0807 15:12:18.250637 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:18.250641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:18.250732 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:18.250801 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:18.252635 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:18.253549 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:18.255374 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:18.255730 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:18.255820 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:18.259348 (+ 3528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:18.259353 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:18.259356 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:18.259386 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:18.261342 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:18.262069 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:18.262073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:18.262075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:18.262174 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:18.262701 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:18.262705 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:18.262707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:18.262961 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:18.263044 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:18.263851 (+ 807us) server_negotiation.cc:299] Negotiation successful
0807 15:12:18.264012 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 15:12:18.266417 1882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:18.247779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34813 (local address 127.15.226.66:42857)
0807 15:12:18.249075 (+ 1296us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:18.249136 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:18.250344 (+ 1208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:18.251269 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:18.251280 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:18.251721 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:18.252442 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:18.252464 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:18.253729 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:18.253733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:18.255214 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:18.255226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:18.256044 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:18.256048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:18.256281 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:18.257082 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:18.257104 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:18.259069 (+ 1965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:18.261502 (+ 2433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:18.261509 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:18.261529 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:18.261932 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:18.262327 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:18.262330 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:18.262333 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:18.262535 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:18.263139 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:18.263147 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:18.263524 (+ 377us) client_negotiation.cc:769] Sending connection context
0807 15:12:18.264758 (+ 1234us) client_negotiation.cc:240] Negotiation successful
0807 15:12:18.265163 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1073,"thread_start_us":199,"threads_started":1}
I20260807 15:12:18.269281 1879 heartbeater.cc:347] Connected to a master server at 127.15.226.126:34813
I20260807 15:12:18.269604 1879 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:18.270300 1879 heartbeater.cc:511] Master 127.15.226.126:34813 requested a full tablet report, sending...
I20260807 15:12:18.271777 1374 ts_manager.cc:194] Registered new tserver with Master: 084f8dc16f124199aba444b8de8c0f33 (127.15.226.66:43217)
I20260807 15:12:18.272701 1374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.66@KRBTEST.COM'} at 127.15.226.66:42857
WARNING: no policy specified for kudu/127.15.226.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
I20260807 15:12:18.299072 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.226.84:32929/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34813
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:12:18 dist-test-slave-3kk6 krb5kdc[658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115537, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for HTTP/127.15.226.84@KRBTEST.COM
W20260807 15:12:18.457958 1887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:18.458380 1887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:18.458441 1887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:18.464903 1887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:18.465016 1887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:18.465106 1887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:12:18.472975 1887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.226.84:32929/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:18.474615 1887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:18.476046 1887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:18.488608 1895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:18.488608 1896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:18.488914 1887 server_base.cc:1034] running on GCE node
W20260807 15:12:18.488636 1898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:18.489652 1887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:18.490388 1887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:18.491592 1887 hybrid_clock.cc:648] HybridClock initialized: now 1786115538491573 us; error 41 us; skew 500 ppm
Aug 07 15:12:18 dist-test-slave-3kk6 krb5kdc[658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115538, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:18.494992 1887 init.cc:377] Logged in from keytab as kudu/127.15.226.67@KRBTEST.COM (short username kudu)
I20260807 15:12:18.495985 1887 webserver.cc:460] Webserver started at http://127.15.226.67:44575/ using document root <none> and password file <none>
I20260807 15:12:18.496675 1887 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:18.496785 1887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:18.497092 1887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 15:12:18 dist-test-slave-3kk6 krb5kdc[658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115538, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for HTTP/127.15.226.84@KRBTEST.COM
I20260807 15:12:18.513821 1887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "9043f37860fa4dde98c868ea5e0be138"
format_stamp: "Formatted at 2026-08-07 15:12:18 on dist-test-slave-3kk6"
server_key: "d8538b65e6fb888b4796125edf220a39"
server_key_iv: "da40d4ddb6433fec62ef15e67959a6ac"
server_key_version: "kuduclusterkey@0"
I20260807 15:12:18.514873 1887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "9043f37860fa4dde98c868ea5e0be138"
format_stamp: "Formatted at 2026-08-07 15:12:18 on dist-test-slave-3kk6"
server_key: "d8538b65e6fb888b4796125edf220a39"
server_key_iv: "da40d4ddb6433fec62ef15e67959a6ac"
server_key_version: "kuduclusterkey@0"
I20260807 15:12:18.520761 1887 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 15:12:18.538414 1905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:18.540302 1887 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 15:12:18.540452 1887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "9043f37860fa4dde98c868ea5e0be138"
format_stamp: "Formatted at 2026-08-07 15:12:18 on dist-test-slave-3kk6"
server_key: "d8538b65e6fb888b4796125edf220a39"
server_key_iv: "da40d4ddb6433fec62ef15e67959a6ac"
server_key_version: "kuduclusterkey@0"
I20260807 15:12:18.540599 1887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:18.560209 1887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:18.564527 1887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:18.564759 1887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:18.565637 1887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:18.566908 1887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:18.567011 1887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:18.567093 1887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:18.567137 1887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:18.582636 1887 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:43901
I20260807 15:12:18.584244 1887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:12:18.589007 2065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:43901 every 8 connection(s)
I20260807 15:12:18.591176 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 1887
I20260807 15:12:18.591354 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:12:18.591684 16265 external_mini_cluster.cc:1428] Setting key f279a14fccd1a2a16dbc3874f5082013
Aug 07 15:12:18 dist-test-slave-3kk6 krb5kdc[658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115538, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:12:18.610575 1690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:18.586571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:32985 (local address 127.15.226.126:34813)
0807 15:12:18.587487 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:18.587492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:18.588922 (+ 1430us) server_negotiation.cc:407] Connection header received
0807 15:12:18.590390 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:18.590394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:18.590459 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:18.590606 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:18.594365 (+ 3759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:18.595307 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:18.597525 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:18.597914 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:18.598000 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:18.603064 (+ 5064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:18.603072 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:18.603079 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:18.603129 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:18.605585 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:18.608398 (+ 2813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:18.608404 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:18.608409 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:18.608512 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:18.608998 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:18.609014 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:18.609019 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:18.609289 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:18.609378 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:18.610165 (+ 787us) server_negotiation.cc:299] Negotiation successful
0807 15:12:18.610324 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781}
I20260807 15:12:18.612808 2071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:18.587108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34813 (local address 127.15.226.67:32985)
0807 15:12:18.588628 (+ 1520us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:18.588726 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:18.590028 (+ 1302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:18.590997 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:18.591009 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:18.591727 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:18.592846 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:18.592939 (+ 93us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:18.595567 (+ 2628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:18.595572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:18.597356 (+ 1784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:18.597370 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:18.598330 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:18.598339 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:18.598699 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:18.599875 (+ 1176us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:18.599903 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:18.602754 (+ 2851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:18.607729 (+ 4975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:18.607740 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:18.607767 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:18.608262 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:18.608629 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:18.608635 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:18.608640 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:18.608879 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:18.609456 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:18.609466 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:18.609863 (+ 397us) client_negotiation.cc:769] Sending connection context
0807 15:12:18.611079 (+ 1216us) client_negotiation.cc:240] Negotiation successful
0807 15:12:18.611493 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1172,"thread_start_us":312,"threads_started":1}
I20260807 15:12:18.614369 2066 heartbeater.cc:347] Connected to a master server at 127.15.226.126:34813
I20260807 15:12:18.614615 2066 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:18.615214 2066 heartbeater.cc:511] Master 127.15.226.126:34813 requested a full tablet report, sending...
I20260807 15:12:18.616354 1374 ts_manager.cc:194] Registered new tserver with Master: 9043f37860fa4dde98c868ea5e0be138 (127.15.226.67:43901)
I20260807 15:12:18.616958 1374 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.67@KRBTEST.COM'} at 127.15.226.67:32985
I20260807 15:12:18.620625 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:12:18.621327 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 1491
I20260807 15:12:18.652454 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 1694
I20260807 15:12:18.678320 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 1887
I20260807 15:12:18.693639 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 1328
I20260807 15:12:18.710992 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:34813
--webserver_interface=127.15.226.126
--webserver_port=40065
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:34813
--ranger_config_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.226.84:32929/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.226.84:32929/kms with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:18.851363 2072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:18.851676 2072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:18.851792 2072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:18.856160 2072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:12:18.856259 2072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:18.856279 2072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:18.856297 2072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:12:18.856313 2072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:12:18.862568 2072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.226.84:32929/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:34813
--ranger_config_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:34813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=40065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:18.863873 2072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:18.864987 2072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:18.874729 2080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:18.874781 2079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:18.874854 2082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:18.874998 2072 server_base.cc:1034] running on GCE node
I20260807 15:12:18.875763 2072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:18.877107 2072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:18.878310 2072 hybrid_clock.cc:648] HybridClock initialized: now 1786115538878267 us; error 59 us; skew 500 ppm
Aug 07 15:12:18 dist-test-slave-3kk6 krb5kdc[658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115538, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:18.881801 2072 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:12:18.883085 2072 webserver.cc:460] Webserver started at http://127.15.226.126:40065/ using document root <none> and password file <none>
I20260807 15:12:18.883680 2072 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:18.883790 2072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 15:12:18 dist-test-slave-3kk6 krb5kdc[658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115538, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for HTTP/127.15.226.84@KRBTEST.COM
I20260807 15:12:18.941633 2072 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260807 15:12:18.944429 2094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:18.945976 2072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:12:18.946165 2072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "6e4b306b689241a3a35a11259cff3810"
format_stamp: "Formatted at 2026-08-07 15:12:16 on dist-test-slave-3kk6"
server_key: "f0f0de41b69554b5fbb7f748e8de6e82"
server_key_iv: "661fe8162628421d30091d0060fcf644"
server_key_version: "kuduclusterkey@0"
I20260807 15:12:18.946852 2072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:18.965924 2072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:18.969007 2072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:18.969251 2072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:18.979445 2072 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:34813
I20260807 15:12:18.979471 2168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:34813 every 8 connection(s)
I20260807 15:12:18.980831 2072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:12:18.983366 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 2072
I20260807 15:12:18.983822 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.226.84:32929/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:42331
--local_ip_for_outbound_sockets=127.15.226.65
--tserver_master_addrs=127.15.226.126:34813
--webserver_port=35715
--webserver_interface=127.15.226.65
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.226.84:32929/kms with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:12:18.985626 2170 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:12:18.987541 2170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810: Bootstrap starting.
I20260807 15:12:19.023094 2170 log.cc:824] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:19.030604 2170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:12:19.031232 2170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810: Bootstrap complete.
I20260807 15:12:19.035669 2170 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4b306b689241a3a35a11259cff3810" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34813 } }
I20260807 15:12:19.036104 2170 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e4b306b689241a3a35a11259cff3810, State: Initialized, Role: FOLLOWER
I20260807 15:12:19.036697 2170 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4b306b689241a3a35a11259cff3810" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34813 } }
I20260807 15:12:19.036829 2170 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:12:19.036904 2170 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:12:19.037010 2170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:12:19.039289 2170 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4b306b689241a3a35a11259cff3810" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34813 } }
I20260807 15:12:19.039661 2170 leader_election.cc:302] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e4b306b689241a3a35a11259cff3810; no voters:
I20260807 15:12:19.040045 2170 leader_election.cc:288] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 15:12:19.040143 2176 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:12:19.040529 2176 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [term 2 LEADER]: Becoming Leader. State: Replica: 6e4b306b689241a3a35a11259cff3810, State: Running, Role: LEADER
I20260807 15:12:19.040926 2176 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4b306b689241a3a35a11259cff3810" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34813 } }
I20260807 15:12:19.041455 2170 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:12:19.042560 2178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6e4b306b689241a3a35a11259cff3810" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4b306b689241a3a35a11259cff3810" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34813 } } }
I20260807 15:12:19.042752 2178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:19.043078 2179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e4b306b689241a3a35a11259cff3810. Latest consensus state: current_term: 2 leader_uuid: "6e4b306b689241a3a35a11259cff3810" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4b306b689241a3a35a11259cff3810" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34813 } } }
I20260807 15:12:19.043176 2179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810 [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:19.043931 2190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:12:19.044472 2170 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 15:12:19.051391 2190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:12:19.052349 2190 catalog_manager.cc:1259] Loaded cluster ID: 2003786bb89d4f1c88885659ec555a15
I20260807 15:12:19.052460 2190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:12:19.055148 2190 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:12:19.055969 2190 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810: Loaded TSK: 0
I20260807 15:12:19.057335 2190 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:12:19.164336 2173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:19.164784 2173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:19.164927 2173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:19.169953 2173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:19.170142 2173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:19.170296 2173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:12:19.176612 2173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.226.84:32929/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:42331
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=35715
--tserver_master_addrs=127.15.226.126:34813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:19.177817 2173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:19.178962 2173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:19.193821 2236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:19.194043 2173 server_base.cc:1034] running on GCE node
W20260807 15:12:19.194525 2232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:19.194789 2231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:19.195513 2173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:19.196404 2173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:19.197621 2173 hybrid_clock.cc:648] HybridClock initialized: now 1786115539197588 us; error 53 us; skew 500 ppm
Aug 07 15:12:19 dist-test-slave-3kk6 krb5kdc[658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115539, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:19.202621 2173 init.cc:377] Logged in from keytab as kudu/127.15.226.65@KRBTEST.COM (short username kudu)
I20260807 15:12:19.204265 2173 webserver.cc:460] Webserver started at http://127.15.226.65:35715/ using document root <none> and password file <none>
I20260807 15:12:19.205168 2173 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:19.205338 2173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 15:12:19 dist-test-slave-3kk6 krb5kdc[658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115539, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for HTTP/127.15.226.84@KRBTEST.COM
I20260807 15:12:19.232792 2173 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.004s sys 0.000s
I20260807 15:12:19.237617 2248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:19.239570 2173 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260807 15:12:19.239741 2173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "7f98b732e2af4a05b09ee79a75cff680"
format_stamp: "Formatted at 2026-08-07 15:12:17 on dist-test-slave-3kk6"
server_key: "a7193fb39b7997ab360e78657b15e3e6"
server_key_iv: "30f28b02f340e6dcf58a5972c5e9f9d9"
server_key_version: "kuduclusterkey@0"
I20260807 15:12:19.240564 2173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:19.256160 2173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:19.260386 2173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:19.260648 2173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:19.261534 2173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:19.262792 2173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:19.262872 2173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:19.262941 2173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:19.262972 2173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:19.277009 2173 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:42331
I20260807 15:12:19.277060 2418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:42331 every 8 connection(s)
I20260807 15:12:19.278508 2173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:12:19.286422 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 2173
I20260807 15:12:19.287031 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.226.84:32929/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:43217
--local_ip_for_outbound_sockets=127.15.226.66
--tserver_master_addrs=127.15.226.126:34813
--webserver_port=36095
--webserver_interface=127.15.226.66
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.226.84:32929/kms with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:12:19 dist-test-slave-3kk6 krb5kdc[658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115539, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:12:19.299988 2423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:19.281369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34813 (local address 127.15.226.65:60059)
0807 15:12:19.281919 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:19.281962 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:19.282933 (+ 971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:19.284189 (+ 1256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:19.284197 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:19.284568 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:19.285034 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:19.285048 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.286474 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.286486 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:19.287669 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:19.287680 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.288513 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.288517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:19.288711 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:19.289506 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:19.289531 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:19.291535 (+ 2004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:19.294898 (+ 3363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:19.294909 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:19.294931 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:19.295425 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:19.296105 (+ 680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:19.296110 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:19.296113 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:19.296394 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:19.297395 (+ 1001us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:19.297404 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:19.297787 (+ 383us) client_negotiation.cc:769] Sending connection context
0807 15:12:19.298025 (+ 238us) client_negotiation.cc:240] Negotiation successful
0807 15:12:19.298498 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":176,"threads_started":1}
I20260807 15:12:19.303510 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:19.281190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:60059 (local address 127.15.226.126:34813)
0807 15:12:19.281700 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:19.281727 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:19.282103 (+ 376us) server_negotiation.cc:407] Connection header received
0807 15:12:19.283394 (+ 1291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:19.283405 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:19.283765 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:19.284113 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:19.285231 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.286331 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.287843 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.288331 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.288521 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:19.291847 (+ 3326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:19.291857 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:19.291881 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:19.291924 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:19.294679 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:19.295644 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:19.295655 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:19.295672 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:19.295894 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:19.296554 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:19.296558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:19.296560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:19.297215 (+ 655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:19.301415 (+ 4200us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:19.301681 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 15:12:19.302125 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":128,"threads_started":1}
I20260807 15:12:19.307580 2419 heartbeater.cc:347] Connected to a master server at 127.15.226.126:34813
I20260807 15:12:19.307909 2419 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:19.308637 2419 heartbeater.cc:511] Master 127.15.226.126:34813 requested a full tablet report, sending...
W20260807 15:12:19.311455 2419 heartbeater.cc:644] Failed to heartbeat to 127.15.226.126:34813 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 15:12:19.311625 2419 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:19.311944 2419 heartbeater.cc:511] Master 127.15.226.126:34813 requested a full tablet report, sending...
I20260807 15:12:19.313177 2419 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:19.313390 2419 heartbeater.cc:511] Master 127.15.226.126:34813 requested a full tablet report, sending...
W20260807 15:12:19.314311 2419 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:12:19.448231 2425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:19.448565 2425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:19.448611 2425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:19.454517 2425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:19.454671 2425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:19.454769 2425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:12:19.463667 2425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.226.84:32929/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:43217
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=36095
--tserver_master_addrs=127.15.226.126:34813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:19.465235 2425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:19.466797 2425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:19.478946 2431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:19.478960 2437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:19.478966 2434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:19.479954 2425 server_base.cc:1034] running on GCE node
I20260807 15:12:19.480501 2425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:19.481398 2425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:19.482610 2425 hybrid_clock.cc:648] HybridClock initialized: now 1786115539482571 us; error 63 us; skew 500 ppm
Aug 07 15:12:19 dist-test-slave-3kk6 krb5kdc[658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115539, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:19.486778 2425 init.cc:377] Logged in from keytab as kudu/127.15.226.66@KRBTEST.COM (short username kudu)
I20260807 15:12:19.488248 2425 webserver.cc:460] Webserver started at http://127.15.226.66:36095/ using document root <none> and password file <none>
I20260807 15:12:19.489065 2425 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:19.489202 2425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 15:12:19 dist-test-slave-3kk6 krb5kdc[658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115539, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for HTTP/127.15.226.84@KRBTEST.COM
I20260807 15:12:19.510878 2425 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 15:12:19.514567 2452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:19.516273 2425 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 15:12:19.516487 2425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "084f8dc16f124199aba444b8de8c0f33"
format_stamp: "Formatted at 2026-08-07 15:12:18 on dist-test-slave-3kk6"
server_key: "5bd980c1803bb79f8c55374441a76ae7"
server_key_iv: "47719a0f09b669ac862ca4c235861c25"
server_key_version: "kuduclusterkey@0"
I20260807 15:12:19.517194 2425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:19.558977 2425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:19.563836 2425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:19.564141 2425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:19.564977 2425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:19.566228 2425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:19.566349 2425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:19.566452 2425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:19.566517 2425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:19.577541 2425 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:43217
I20260807 15:12:19.579087 2425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:12:19.580878 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 2425
I20260807 15:12:19.581367 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.226.84:32929/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:43901
--local_ip_for_outbound_sockets=127.15.226.67
--tserver_master_addrs=127.15.226.126:34813
--webserver_port=44575
--webserver_interface=127.15.226.67
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:34981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.226.84:32929/kms with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:12:19.585604 2604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:43217 every 8 connection(s)
Aug 07 15:12:19 dist-test-slave-3kk6 krb5kdc[658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115539, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:12:19.605163 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:19.589988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:51705 (local address 127.15.226.126:34813)
0807 15:12:19.590128 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:19.590134 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:19.590157 (+ 23us) server_negotiation.cc:407] Connection header received
0807 15:12:19.591263 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:19.591267 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:19.591334 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:19.591445 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:19.593270 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.594231 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.596458 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.596927 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.597038 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:19.600562 (+ 3524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:19.600567 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:19.600570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:19.600602 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:19.602473 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:19.603227 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:19.603231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:19.603233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:19.603326 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:19.603735 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:19.603739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:19.603740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:19.603981 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:19.604066 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:19.604801 (+ 735us) server_negotiation.cc:299] Negotiation successful
0807 15:12:19.604957 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 15:12:19.607317 2611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:19.588656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34813 (local address 127.15.226.66:51705)
0807 15:12:19.589362 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:19.589438 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:19.590919 (+ 1481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:19.591726 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:19.591740 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:19.592168 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:19.593013 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:19.593043 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.594458 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.594463 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:19.596264 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:19.596279 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.597314 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.597321 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:19.597577 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:19.598384 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:19.598409 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:19.600328 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:19.602628 (+ 2300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:19.602638 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:19.602700 (+ 62us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:19.603097 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:19.603434 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:19.603436 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:19.603438 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:19.603619 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:19.604135 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:19.604143 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:19.604488 (+ 345us) client_negotiation.cc:769] Sending connection context
0807 15:12:19.605753 (+ 1265us) client_negotiation.cc:240] Negotiation successful
0807 15:12:19.606151 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":198,"threads_started":1}
I20260807 15:12:19.609050 2607 heartbeater.cc:347] Connected to a master server at 127.15.226.126:34813
I20260807 15:12:19.609306 2607 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:19.609935 2607 heartbeater.cc:511] Master 127.15.226.126:34813 requested a full tablet report, sending...
W20260807 15:12:19.611855 2607 heartbeater.cc:644] Failed to heartbeat to 127.15.226.126:34813 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 15:12:19.611984 2607 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:19.612165 2607 heartbeater.cc:511] Master 127.15.226.126:34813 requested a full tablet report, sending...
I20260807 15:12:19.613095 2607 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:19.613263 2607 heartbeater.cc:511] Master 127.15.226.126:34813 requested a full tablet report, sending...
W20260807 15:12:19.614037 2607 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:12:19.739431 2609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:19.739895 2609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:19.739976 2609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:19.746564 2609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:19.746753 2609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:19.746889 2609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:12:19.755470 2609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:34981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.226.84:32929/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:43901
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=44575
--tserver_master_addrs=127.15.226.126:34813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:19.757023 2609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:19.758467 2609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:19.772480 2622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:19.772516 2626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:19.772516 2624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:19.773129 2609 server_base.cc:1034] running on GCE node
I20260807 15:12:19.773669 2609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:19.774473 2609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:19.775691 2609 hybrid_clock.cc:648] HybridClock initialized: now 1786115539775659 us; error 57 us; skew 500 ppm
Aug 07 15:12:19 dist-test-slave-3kk6 krb5kdc[658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115539, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:19.779070 2609 init.cc:377] Logged in from keytab as kudu/127.15.226.67@KRBTEST.COM (short username kudu)
I20260807 15:12:19.780328 2609 webserver.cc:460] Webserver started at http://127.15.226.67:44575/ using document root <none> and password file <none>
I20260807 15:12:19.781049 2609 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:19.781183 2609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 15:12:19 dist-test-slave-3kk6 krb5kdc[658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115539, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for HTTP/127.15.226.84@KRBTEST.COM
I20260807 15:12:19.814086 2609 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.003s
I20260807 15:12:19.817009 2637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:19.818487 2609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 15:12:19.818663 2609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "9043f37860fa4dde98c868ea5e0be138"
format_stamp: "Formatted at 2026-08-07 15:12:18 on dist-test-slave-3kk6"
server_key: "d8538b65e6fb888b4796125edf220a39"
server_key_iv: "da40d4ddb6433fec62ef15e67959a6ac"
server_key_version: "kuduclusterkey@0"
I20260807 15:12:19.819525 2609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:19.844228 2609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:19.848361 2609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:19.848595 2609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:19.849315 2609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:19.850322 2609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:19.850395 2609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:19.850467 2609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:19.850504 2609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:19.863958 2609 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:43901
I20260807 15:12:19.865425 2609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:12:19.867401 2788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:43901 every 8 connection(s)
I20260807 15:12:19.874159 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 2609
Aug 07 15:12:19 dist-test-slave-3kk6 krb5kdc[658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115539, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:12:19.888579 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:19.869505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:60143 (local address 127.15.226.126:34813)
0807 15:12:19.869656 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:19.869661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:19.870628 (+ 967us) server_negotiation.cc:407] Connection header received
0807 15:12:19.872008 (+ 1380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:19.872012 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:19.872111 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:19.872176 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:19.874421 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.875297 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.877660 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.877992 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.878124 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:19.883335 (+ 5211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:19.883339 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:19.883342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:19.883377 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:19.885474 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:19.886203 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:19.886208 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:19.886210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:19.886303 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:19.887050 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:19.887054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:19.887056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:19.887322 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:19.887451 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:19.888225 (+ 774us) server_negotiation.cc:299] Negotiation successful
0807 15:12:19.888384 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 15:12:19.890200 2794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:19.876288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36222 (local address 127.15.226.126:34813)
0807 15:12:19.878864 (+ 2576us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:19.878887 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:19.878908 (+ 21us) server_negotiation.cc:407] Connection header received
0807 15:12:19.878955 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:19.878959 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:19.879013 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:19.879117 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:19.880196 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.881012 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.883196 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.883495 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.883617 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:19.885627 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:19.885630 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:19.885633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:19.885659 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:19.887615 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:19.888195 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:19.888198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:19.888200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:19.888278 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:19.888600 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:19.888605 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:19.888607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:19.889002 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:19.889203 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:19.889920 (+ 717us) server_negotiation.cc:299] Negotiation successful
0807 15:12:19.890047 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2495,"thread_start_us":132,"threads_started":1}
I20260807 15:12:19.890677 2792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:19.869902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34813 (local address 127.15.226.67:60143)
0807 15:12:19.870445 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:19.870503 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:19.871708 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:19.872826 (+ 1118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:19.872837 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:19.873253 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:19.874194 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:19.874219 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.876321 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.876324 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:19.877509 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:19.877521 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:19.878213 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:19.878221 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:19.878461 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:19.880328 (+ 1867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:19.880353 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:19.883037 (+ 2684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:19.885618 (+ 2581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:19.885625 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:19.885647 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:19.886084 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:19.886403 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:19.886408 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:19.886413 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:19.886637 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:19.887469 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:19.887479 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:19.887893 (+ 414us) client_negotiation.cc:769] Sending connection context
0807 15:12:19.888907 (+ 1014us) client_negotiation.cc:240] Negotiation successful
0807 15:12:19.889305 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":188,"threads_started":1}
I20260807 15:12:19.892153 2789 heartbeater.cc:347] Connected to a master server at 127.15.226.126:34813
I20260807 15:12:19.892434 2789 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:19.893051 2789 heartbeater.cc:511] Master 127.15.226.126:34813 requested a full tablet report, sending...
W20260807 15:12:19.894834 2789 heartbeater.cc:644] Failed to heartbeat to 127.15.226.126:34813 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 15:12:19.894984 2789 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:19.895187 2789 heartbeater.cc:511] Master 127.15.226.126:34813 requested a full tablet report, sending...
I20260807 15:12:19.896088 2789 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:19.896297 2789 heartbeater.cc:511] Master 127.15.226.126:34813 requested a full tablet report, sending...
W20260807 15:12:19.897145 2789 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 15:12:20 dist-test-slave-3kk6 krb5kdc[658](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786115540, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:20.314587 2419 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:20.314960 2419 heartbeater.cc:511] Master 127.15.226.126:34813 requested a full tablet report, sending...
I20260807 15:12:20.316215 2116 ts_manager.cc:194] Registered new tserver with Master: 7f98b732e2af4a05b09ee79a75cff680 (127.15.226.65:42331)
I20260807 15:12:20.318033 2116 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.65@KRBTEST.COM'} at 127.15.226.65:60059
I20260807 15:12:20.614317 2607 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:20.614766 2607 heartbeater.cc:511] Master 127.15.226.126:34813 requested a full tablet report, sending...
I20260807 15:12:20.615803 2116 ts_manager.cc:194] Registered new tserver with Master: 084f8dc16f124199aba444b8de8c0f33 (127.15.226.66:43217)
I20260807 15:12:20.616691 2116 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.66@KRBTEST.COM'} at 127.15.226.66:51705
Aug 07 15:12:20 dist-test-slave-3kk6 krb5kdc[658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115540, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for HTTP/127.15.226.84@KRBTEST.COM
I20260807 15:12:20.897423 2789 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:20.897765 2789 heartbeater.cc:511] Master 127.15.226.126:34813 requested a full tablet report, sending...
I20260807 15:12:20.898583 2116 ts_manager.cc:194] Registered new tserver with Master: 9043f37860fa4dde98c868ea5e0be138 (127.15.226.67:43901)
I20260807 15:12:20.899432 2116 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.67@KRBTEST.COM'} at 127.15.226.67:60143
I20260807 15:12:20.904474 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:12:20.920645 2821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:20.906328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36238 (local address 127.15.226.126:34813)
0807 15:12:20.906963 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:20.906968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:20.907327 (+ 359us) server_negotiation.cc:407] Connection header received
0807 15:12:20.907372 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:20.907375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:20.907422 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:20.907499 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:20.908520 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:20.909380 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:20.910601 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:20.910985 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:20.911093 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:20.916334 (+ 5241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:20.916338 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:20.916341 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:20.916383 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:20.918486 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:20.919094 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:20.919098 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:20.919100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:20.919207 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:20.919574 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:20.919578 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:20.919580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:20.919848 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:20.920000 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:20.920286 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 15:12:20.920443 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":89,"threads_started":1}
I20260807 15:12:20.925663 2116 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36238:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:12:20.930586 2116 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:12:20.958356 2829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:20.946805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:43901 (local address 127.0.0.1:38564)
0807 15:12:20.952351 (+ 5546us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:20.952368 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:20.952488 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:20.953305 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:20.953308 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:20.953324 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:12:20.953433 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:20.953439 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:20.954900 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:20.954903 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:20.956379 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:20.956391 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:20.957769 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:20.957773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:20.957889 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:20.957929 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 15:12:20.958039 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 15:12:20.958165 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5475,"mutex_wait_us":19,"thread_start_us":118,"threads_started":1}
I20260807 15:12:20.959031 2830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:20.947279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38564 (local address 127.15.226.67:43901)
0807 15:12:20.948255 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:20.948263 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:20.952607 (+ 4344us) server_negotiation.cc:407] Connection header received
0807 15:12:20.952701 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:20.952706 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:20.952897 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:20.953111 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:12:20.953561 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:20.954764 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:20.956538 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:20.957423 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:20.957532 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:20.957695 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:20.958774 (+ 1079us) server_negotiation.cc:299] Negotiation successful
0807 15:12:20.958888 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":860,"thread_start_us":115,"threads_started":1}
I20260807 15:12:20.961896 2706 tablet_service.cc:1467] Processing CreateTablet for tablet ad760eb9c2ec4472b55cd05908d28350 (DEFAULT_TABLE table=test-table [id=bc52d4b060f349bf876bf20d8eb6d98a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:12:20.963083 2706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad760eb9c2ec4472b55cd05908d28350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:20.967775 2827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:20.945471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:42331 (local address 127.0.0.1:39742)
0807 15:12:20.951534 (+ 6063us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:20.951603 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:20.951776 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:20.952610 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:20.952617 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:20.952640 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:12:20.952780 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:20.952787 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:20.955095 (+ 2308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:20.955098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:20.965713 (+ 10615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:20.965729 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:20.967305 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:20.967309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:20.967426 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:20.967451 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 15:12:20.967524 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 15:12:20.967610 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5852,"mutex_wait_us":78,"thread_start_us":131,"threads_started":1}
I20260807 15:12:20.968851 2828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:20.945588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39742 (local address 127.15.226.65:42331)
0807 15:12:20.951210 (+ 5622us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:20.951218 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:20.951926 (+ 708us) server_negotiation.cc:407] Connection header received
0807 15:12:20.952011 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:20.952016 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:20.952253 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:20.952412 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:12:20.952911 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:20.954965 (+ 2054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:20.965909 (+ 10944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:20.966858 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:20.966956 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:20.967121 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:20.968581 (+ 1460us) server_negotiation.cc:299] Negotiation successful
0807 15:12:20.968703 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5453,"thread_start_us":107,"threads_started":1}
I20260807 15:12:20.970402 2831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:20.947801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:43217 (local address 127.0.0.1:44376)
0807 15:12:20.953227 (+ 5426us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:20.953242 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:20.953343 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:20.955285 (+ 1942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:20.955288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:20.955306 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:12:20.955382 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:20.955387 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:20.967253 (+ 11866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:20.967257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:20.968437 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:20.968449 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:20.969953 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:20.969957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:20.970058 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:20.970085 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 15:12:20.970162 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 15:12:20.970250 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5366,"mutex_wait_us":1,"spinlock_wait_cycles":768,"thread_start_us":100,"threads_started":1}
I20260807 15:12:20.970911 2834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:20.947974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44376 (local address 127.15.226.66:43217)
0807 15:12:20.954705 (+ 6731us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:20.954722 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:20.954749 (+ 27us) server_negotiation.cc:407] Connection header received
0807 15:12:20.954833 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:20.954841 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:20.955033 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:20.955171 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:12:20.957424 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:20.967079 (+ 9655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:20.968747 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:20.969608 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:20.969721 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:20.969872 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:20.970611 (+ 739us) server_negotiation.cc:299] Negotiation successful
0807 15:12:20.970767 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5078,"thread_start_us":107,"threads_started":1}
I20260807 15:12:20.972194 2296 tablet_service.cc:1467] Processing CreateTablet for tablet ad760eb9c2ec4472b55cd05908d28350 (DEFAULT_TABLE table=test-table [id=bc52d4b060f349bf876bf20d8eb6d98a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:12:20.973366 2296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad760eb9c2ec4472b55cd05908d28350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:20.973836 2509 tablet_service.cc:1467] Processing CreateTablet for tablet ad760eb9c2ec4472b55cd05908d28350 (DEFAULT_TABLE table=test-table [id=bc52d4b060f349bf876bf20d8eb6d98a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:12:20.974874 2509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad760eb9c2ec4472b55cd05908d28350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:20.982314 2838 tablet_bootstrap.cc:492] T ad760eb9c2ec4472b55cd05908d28350 P 7f98b732e2af4a05b09ee79a75cff680: Bootstrap starting.
I20260807 15:12:20.984838 2838 tablet_bootstrap.cc:654] T ad760eb9c2ec4472b55cd05908d28350 P 7f98b732e2af4a05b09ee79a75cff680: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:20.985387 2836 tablet_bootstrap.cc:492] T ad760eb9c2ec4472b55cd05908d28350 P 9043f37860fa4dde98c868ea5e0be138: Bootstrap starting.
I20260807 15:12:20.985766 2838 log.cc:824] T ad760eb9c2ec4472b55cd05908d28350 P 7f98b732e2af4a05b09ee79a75cff680: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:20.988243 2836 tablet_bootstrap.cc:654] T ad760eb9c2ec4472b55cd05908d28350 P 9043f37860fa4dde98c868ea5e0be138: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:20.988708 2838 tablet_bootstrap.cc:492] T ad760eb9c2ec4472b55cd05908d28350 P 7f98b732e2af4a05b09ee79a75cff680: No bootstrap required, opened a new log
I20260807 15:12:20.988935 2838 ts_tablet_manager.cc:1397] T ad760eb9c2ec4472b55cd05908d28350 P 7f98b732e2af4a05b09ee79a75cff680: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 15:12:20.993352 2836 log.cc:824] T ad760eb9c2ec4472b55cd05908d28350 P 9043f37860fa4dde98c868ea5e0be138: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:20.994153 2838 raft_consensus.cc:348] T ad760eb9c2ec4472b55cd05908d28350 P 7f98b732e2af4a05b09ee79a75cff680 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f98b732e2af4a05b09ee79a75cff680" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42331 } } peers { permanent_uuid: "9043f37860fa4dde98c868ea5e0be138" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43901 } } peers { permanent_uuid: "084f8dc16f124199aba444b8de8c0f33" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43217 } }
I20260807 15:12:20.994460 2838 raft_consensus.cc:374] T ad760eb9c2ec4472b55cd05908d28350 P 7f98b732e2af4a05b09ee79a75cff680 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:20.994532 2838 raft_consensus.cc:729] T ad760eb9c2ec4472b55cd05908d28350 P 7f98b732e2af4a05b09ee79a75cff680 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f98b732e2af4a05b09ee79a75cff680, State: Initialized, Role: FOLLOWER
I20260807 15:12:20.995112 2838 consensus_queue.cc:260] T ad760eb9c2ec4472b55cd05908d28350 P 7f98b732e2af4a05b09ee79a75cff680 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f98b732e2af4a05b09ee79a75cff680" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42331 } } peers { permanent_uuid: "9043f37860fa4dde98c868ea5e0be138" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43901 } } peers { permanent_uuid: "084f8dc16f124199aba444b8de8c0f33" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43217 } }
I20260807 15:12:20.996207 2838 ts_tablet_manager.cc:1428] T ad760eb9c2ec4472b55cd05908d28350 P 7f98b732e2af4a05b09ee79a75cff680: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260807 15:12:20.996567 2419 heartbeater.cc:503] Master 127.15.226.126:34813 was elected leader, sending a full tablet report...
I20260807 15:12:20.999526 2842 tablet_bootstrap.cc:492] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33: Bootstrap starting.
Aug 07 15:12:21 dist-test-slave-3kk6 krb5kdc[658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115540, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for HTTP/127.15.226.84@KRBTEST.COM
I20260807 15:12:21.004199 2836 tablet_bootstrap.cc:492] T ad760eb9c2ec4472b55cd05908d28350 P 9043f37860fa4dde98c868ea5e0be138: No bootstrap required, opened a new log
I20260807 15:12:21.004431 2836 ts_tablet_manager.cc:1397] T ad760eb9c2ec4472b55cd05908d28350 P 9043f37860fa4dde98c868ea5e0be138: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.000s
I20260807 15:12:21.008710 2836 raft_consensus.cc:348] T ad760eb9c2ec4472b55cd05908d28350 P 9043f37860fa4dde98c868ea5e0be138 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f98b732e2af4a05b09ee79a75cff680" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42331 } } peers { permanent_uuid: "9043f37860fa4dde98c868ea5e0be138" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43901 } } peers { permanent_uuid: "084f8dc16f124199aba444b8de8c0f33" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43217 } }
I20260807 15:12:21.009011 2842 tablet_bootstrap.cc:654] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:21.009037 2836 raft_consensus.cc:374] T ad760eb9c2ec4472b55cd05908d28350 P 9043f37860fa4dde98c868ea5e0be138 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:21.009104 2836 raft_consensus.cc:729] T ad760eb9c2ec4472b55cd05908d28350 P 9043f37860fa4dde98c868ea5e0be138 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9043f37860fa4dde98c868ea5e0be138, State: Initialized, Role: FOLLOWER
I20260807 15:12:21.009675 2836 consensus_queue.cc:260] T ad760eb9c2ec4472b55cd05908d28350 P 9043f37860fa4dde98c868ea5e0be138 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f98b732e2af4a05b09ee79a75cff680" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42331 } } peers { permanent_uuid: "9043f37860fa4dde98c868ea5e0be138" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43901 } } peers { permanent_uuid: "084f8dc16f124199aba444b8de8c0f33" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43217 } }
I20260807 15:12:21.010027 2842 log.cc:824] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:21.010803 2836 ts_tablet_manager.cc:1428] T ad760eb9c2ec4472b55cd05908d28350 P 9043f37860fa4dde98c868ea5e0be138: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260807 15:12:21.011188 2789 heartbeater.cc:503] Master 127.15.226.126:34813 was elected leader, sending a full tablet report...
I20260807 15:12:21.016028 2842 tablet_bootstrap.cc:492] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33: No bootstrap required, opened a new log
I20260807 15:12:21.016288 2842 ts_tablet_manager.cc:1397] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.000s
I20260807 15:12:21.020720 2842 raft_consensus.cc:348] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f98b732e2af4a05b09ee79a75cff680" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42331 } } peers { permanent_uuid: "9043f37860fa4dde98c868ea5e0be138" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43901 } } peers { permanent_uuid: "084f8dc16f124199aba444b8de8c0f33" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43217 } }
I20260807 15:12:21.021064 2842 raft_consensus.cc:374] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:21.021133 2842 raft_consensus.cc:729] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 084f8dc16f124199aba444b8de8c0f33, State: Initialized, Role: FOLLOWER
I20260807 15:12:21.022539 2842 consensus_queue.cc:260] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f98b732e2af4a05b09ee79a75cff680" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42331 } } peers { permanent_uuid: "9043f37860fa4dde98c868ea5e0be138" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43901 } } peers { permanent_uuid: "084f8dc16f124199aba444b8de8c0f33" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43217 } }
I20260807 15:12:21.024025 2842 ts_tablet_manager.cc:1428] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260807 15:12:21.024392 2607 heartbeater.cc:503] Master 127.15.226.126:34813 was elected leader, sending a full tablet report...
W20260807 15:12:21.031054 2420 tablet.cc:2369] T ad760eb9c2ec4472b55cd05908d28350 P 7f98b732e2af4a05b09ee79a75cff680: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:12:21.091913 2608 tablet.cc:2369] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:12:21.120373 2791 tablet.cc:2369] T ad760eb9c2ec4472b55cd05908d28350 P 9043f37860fa4dde98c868ea5e0be138: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:12:21.293560 2845 raft_consensus.cc:482] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:12:21.293833 2845 raft_consensus.cc:504] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f98b732e2af4a05b09ee79a75cff680" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42331 } } peers { permanent_uuid: "9043f37860fa4dde98c868ea5e0be138" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43901 } } peers { permanent_uuid: "084f8dc16f124199aba444b8de8c0f33" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43217 } }
I20260807 15:12:21.295514 2845 leader_election.cc:288] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f98b732e2af4a05b09ee79a75cff680 (127.15.226.65:42331), 9043f37860fa4dde98c868ea5e0be138 (127.15.226.67:43901)
I20260807 15:12:21.300323 2830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:21.296016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:32851 (local address 127.15.226.67:43901)
0807 15:12:21.296186 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:21.296191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:21.296395 (+ 204us) server_negotiation.cc:407] Connection header received
0807 15:12:21.296599 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:21.296605 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:21.296666 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:21.296757 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:12:21.297087 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:21.297867 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:21.299297 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:21.299659 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:21.299823 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:21.299883 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:21.300085 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 15:12:21.300169 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 15:12:21.300433 2856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:21.295782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:42331 (local address 127.15.226.66:57167)
0807 15:12:21.296383 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:21.296400 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:21.296542 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:21.296902 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:21.296907 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:21.296923 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:12:21.297033 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:21.297041 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:21.297754 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:21.297757 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:21.299119 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:21.299131 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:21.299987 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:21.299994 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:21.300104 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:21.300123 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 15:12:21.300194 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 15:12:21.300262 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"mutex_wait_us":169,"thread_start_us":73,"threads_started":1}
I20260807 15:12:21.300446 2828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:21.295926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:57167 (local address 127.15.226.65:42331)
0807 15:12:21.296071 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:21.296076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:21.296500 (+ 424us) server_negotiation.cc:407] Connection header received
0807 15:12:21.296688 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:21.296692 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:21.296744 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:21.296830 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:12:21.297132 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:21.297636 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:21.299267 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:21.299878 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:21.300010 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:21.300062 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:21.300228 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 15:12:21.300311 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:12:21.300781 2857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:21.295782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:43901 (local address 127.15.226.66:32851)
0807 15:12:21.296276 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:21.296292 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:21.296437 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:21.296768 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:21.296772 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:21.296787 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:12:21.296953 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:21.296960 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:21.297984 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:21.297987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:21.299141 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:21.299148 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:21.299824 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:21.299828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:21.299931 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:21.299962 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 15:12:21.300555 (+ 593us) client_negotiation.cc:240] Negotiation successful
0807 15:12:21.300637 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":140,"threads_started":1}
I20260807 15:12:21.301301 2352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad760eb9c2ec4472b55cd05908d28350" candidate_uuid: "084f8dc16f124199aba444b8de8c0f33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f98b732e2af4a05b09ee79a75cff680" is_pre_election: true
I20260807 15:12:21.301313 2732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad760eb9c2ec4472b55cd05908d28350" candidate_uuid: "084f8dc16f124199aba444b8de8c0f33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9043f37860fa4dde98c868ea5e0be138" is_pre_election: true
I20260807 15:12:21.301652 2352 raft_consensus.cc:2454] T ad760eb9c2ec4472b55cd05908d28350 P 7f98b732e2af4a05b09ee79a75cff680 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 084f8dc16f124199aba444b8de8c0f33 in term 0.
I20260807 15:12:21.301652 2732 raft_consensus.cc:2454] T ad760eb9c2ec4472b55cd05908d28350 P 9043f37860fa4dde98c868ea5e0be138 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 084f8dc16f124199aba444b8de8c0f33 in term 0.
I20260807 15:12:21.302170 2454 leader_election.cc:302] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 084f8dc16f124199aba444b8de8c0f33, 9043f37860fa4dde98c868ea5e0be138; no voters:
I20260807 15:12:21.302595 2845 raft_consensus.cc:2781] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:12:21.302726 2845 raft_consensus.cc:482] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:12:21.302783 2845 raft_consensus.cc:3037] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:21.304275 2845 raft_consensus.cc:504] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f98b732e2af4a05b09ee79a75cff680" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42331 } } peers { permanent_uuid: "9043f37860fa4dde98c868ea5e0be138" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43901 } } peers { permanent_uuid: "084f8dc16f124199aba444b8de8c0f33" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43217 } }
I20260807 15:12:21.304666 2845 leader_election.cc:288] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33 [CANDIDATE]: Term 1 election: Requested vote from peers 7f98b732e2af4a05b09ee79a75cff680 (127.15.226.65:42331), 9043f37860fa4dde98c868ea5e0be138 (127.15.226.67:43901)
I20260807 15:12:21.305107 2732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad760eb9c2ec4472b55cd05908d28350" candidate_uuid: "084f8dc16f124199aba444b8de8c0f33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9043f37860fa4dde98c868ea5e0be138"
I20260807 15:12:21.305284 2732 raft_consensus.cc:3037] T ad760eb9c2ec4472b55cd05908d28350 P 9043f37860fa4dde98c868ea5e0be138 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:21.306818 2732 raft_consensus.cc:2454] T ad760eb9c2ec4472b55cd05908d28350 P 9043f37860fa4dde98c868ea5e0be138 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 084f8dc16f124199aba444b8de8c0f33 in term 1.
I20260807 15:12:21.307123 2454 leader_election.cc:302] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 084f8dc16f124199aba444b8de8c0f33, 9043f37860fa4dde98c868ea5e0be138; no voters:
I20260807 15:12:21.307097 2352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad760eb9c2ec4472b55cd05908d28350" candidate_uuid: "084f8dc16f124199aba444b8de8c0f33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f98b732e2af4a05b09ee79a75cff680"
I20260807 15:12:21.307247 2352 raft_consensus.cc:3037] T ad760eb9c2ec4472b55cd05908d28350 P 7f98b732e2af4a05b09ee79a75cff680 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:21.307286 2845 raft_consensus.cc:2781] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:12:21.307543 2845 raft_consensus.cc:686] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33 [term 1 LEADER]: Becoming Leader. State: Replica: 084f8dc16f124199aba444b8de8c0f33, State: Running, Role: LEADER
I20260807 15:12:21.307915 2845 consensus_queue.cc:237] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f98b732e2af4a05b09ee79a75cff680" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42331 } } peers { permanent_uuid: "9043f37860fa4dde98c868ea5e0be138" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43901 } } peers { permanent_uuid: "084f8dc16f124199aba444b8de8c0f33" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43217 } }
I20260807 15:12:21.308468 2352 raft_consensus.cc:2454] T ad760eb9c2ec4472b55cd05908d28350 P 7f98b732e2af4a05b09ee79a75cff680 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 084f8dc16f124199aba444b8de8c0f33 in term 1.
I20260807 15:12:21.311915 2116 catalog_manager.cc:5495] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33 reported cstate change: term changed from 0 to 1, leader changed from <none> to 084f8dc16f124199aba444b8de8c0f33 (127.15.226.66). New cstate: current_term: 1 leader_uuid: "084f8dc16f124199aba444b8de8c0f33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f98b732e2af4a05b09ee79a75cff680" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9043f37860fa4dde98c868ea5e0be138" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "084f8dc16f124199aba444b8de8c0f33" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43217 } health_report { overall_health: HEALTHY } } }
I20260807 15:12:21.407758 2834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:21.403699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44384 (local address 127.15.226.66:43217)
0807 15:12:21.403860 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:21.403867 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:21.403884 (+ 17us) server_negotiation.cc:407] Connection header received
0807 15:12:21.404009 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:21.404015 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:21.404087 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:21.404223 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:12:21.404497 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:21.405149 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:21.406052 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:21.406281 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:21.406412 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:21.406784 (+ 372us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:12:21.407310 (+ 526us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:12:21.407455 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:21.407517 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 15:12:21.407577 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 15:12:21.413998 2352 raft_consensus.cc:1261] T ad760eb9c2ec4472b55cd05908d28350 P 7f98b732e2af4a05b09ee79a75cff680 [term 1 FOLLOWER]: Refusing update from remote peer 084f8dc16f124199aba444b8de8c0f33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:12:21.414040 2732 raft_consensus.cc:1261] T ad760eb9c2ec4472b55cd05908d28350 P 9043f37860fa4dde98c868ea5e0be138 [term 1 FOLLOWER]: Refusing update from remote peer 084f8dc16f124199aba444b8de8c0f33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:12:21.414850 2845 consensus_queue.cc:1035] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9043f37860fa4dde98c868ea5e0be138" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 43901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:12:21.415089 2858 consensus_queue.cc:1035] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f98b732e2af4a05b09ee79a75cff680" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:12:21.425482 2862 mvcc.cc:205] Tried to move back new op lower bound from 7315929257623781376 to 7315929257201344512. Current Snapshot: MvccSnapshot[applied={T|T < 7315929257623781376}]
I20260807 15:12:21.426148 2864 mvcc.cc:205] Tried to move back new op lower bound from 7315929257623781376 to 7315929257201344512. Current Snapshot: MvccSnapshot[applied={T|T < 7315929257623781376}]
I20260807 15:12:21.430307 2867 mvcc.cc:205] Tried to move back new op lower bound from 7315929257623781376 to 7315929257201344512. Current Snapshot: MvccSnapshot[applied={T|T < 7315929257623781376}]
I20260807 15:12:21.438298 2115 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:12:21.438505 2115 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:12:21.441597 2115 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e4b306b689241a3a35a11259cff3810: Sending DeleteTablet for 3 replicas of tablet ad760eb9c2ec4472b55cd05908d28350
I20260807 15:12:21.442746 2706 tablet_service.cc:1514] Processing DeleteTablet for tablet ad760eb9c2ec4472b55cd05908d28350 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:12:21 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:38564
I20260807 15:12:21.442740 2296 tablet_service.cc:1514] Processing DeleteTablet for tablet ad760eb9c2ec4472b55cd05908d28350 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:12:21 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:39742
I20260807 15:12:21.443563 2509 tablet_service.cc:1514] Processing DeleteTablet for tablet ad760eb9c2ec4472b55cd05908d28350 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:12:21 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:44376
I20260807 15:12:21.444072 2879 tablet_replica.cc:330] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33: stopping tablet replica
I20260807 15:12:21.444340 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 2173
I20260807 15:12:21.446962 2879 raft_consensus.cc:2229] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:12:21.447530 2879 raft_consensus.cc:2258] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:12:21.450193 2879 ts_tablet_manager.cc:1905] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:12:21.453578 2879 ts_tablet_manager.cc:1918] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:12:21.453693 2879 log.cc:1196] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/ad760eb9c2ec4472b55cd05908d28350
I20260807 15:12:21.454066 2879 ts_tablet_manager.cc:1939] T ad760eb9c2ec4472b55cd05908d28350 P 084f8dc16f124199aba444b8de8c0f33: Deleting consensus metadata
I20260807 15:12:21.460642 2095 catalog_manager.cc:4841] TS 084f8dc16f124199aba444b8de8c0f33 (127.15.226.66:43217): tablet ad760eb9c2ec4472b55cd05908d28350 (table test-table [id=bc52d4b060f349bf876bf20d8eb6d98a]) successfully deleted
I20260807 15:12:21.462981 2876 tablet_replica.cc:330] T ad760eb9c2ec4472b55cd05908d28350 P 9043f37860fa4dde98c868ea5e0be138: stopping tablet replica
I20260807 15:12:21.463514 2876 raft_consensus.cc:2229] T ad760eb9c2ec4472b55cd05908d28350 P 9043f37860fa4dde98c868ea5e0be138 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:12:21.463989 2876 raft_consensus.cc:2258] T ad760eb9c2ec4472b55cd05908d28350 P 9043f37860fa4dde98c868ea5e0be138 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:12:21.465726 2876 ts_tablet_manager.cc:1905] T ad760eb9c2ec4472b55cd05908d28350 P 9043f37860fa4dde98c868ea5e0be138: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:12:21.469183 2876 ts_tablet_manager.cc:1918] T ad760eb9c2ec4472b55cd05908d28350 P 9043f37860fa4dde98c868ea5e0be138: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:12:21.469576 2876 log.cc:1196] T ad760eb9c2ec4472b55cd05908d28350 P 9043f37860fa4dde98c868ea5e0be138: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/ad760eb9c2ec4472b55cd05908d28350
I20260807 15:12:21.470039 2876 ts_tablet_manager.cc:1939] T ad760eb9c2ec4472b55cd05908d28350 P 9043f37860fa4dde98c868ea5e0be138: Deleting consensus metadata
W20260807 15:12:21.471226 2095 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.226.65:42331 (error 108)
I20260807 15:12:21.471765 2096 catalog_manager.cc:4841] TS 9043f37860fa4dde98c868ea5e0be138 (127.15.226.67:43901): tablet ad760eb9c2ec4472b55cd05908d28350 (table test-table [id=bc52d4b060f349bf876bf20d8eb6d98a]) successfully deleted
I20260807 15:12:21.472278 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 2425
I20260807 15:12:21.483428 2883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:21.482847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:42331 (local address 127.0.0.1:39746)
0807 15:12:21.483155 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:21.483265 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.65:42331: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":84,"threads_started":1}
W20260807 15:12:21.483759 2095 catalog_manager.cc:4584] TS 7f98b732e2af4a05b09ee79a75cff680 (127.15.226.65:42331): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ad760eb9c2ec4472b55cd05908d28350: Network error: Client connection negotiation failed: client connection to 127.15.226.65:42331: connect: Connection refused (error 111)
I20260807 15:12:21.488101 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 2609
I20260807 15:12:21.507615 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 2072
I20260807 15:12:21.524245 16265 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 15:12:21.869938 16265 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 15:12:21.870069 16265 mini_ranger.cc:66] Stopping Ranger...
I20260807 15:12:22.215790 16265 mini_ranger.cc:68] Stopped Ranger
2026-08-07 15:12:22.216 UTC [704] LOG: received smart shutdown request
2026-08-07 15:12:22.221 UTC [704] LOG: background worker "logical replication launcher" (PID 722) exited with exit code 1
2026-08-07 15:12:22.221 UTC [710] LOG: shutting down
2026-08-07 15:12:22.396 UTC [704] LOG: database system is shut down
2026-08-07T15:12:22Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53093 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:12:22 dist-test-slave-3kk6 krb5kdc[2893](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:12:22 dist-test-slave-3kk6 krb5kdc[2893](info): set up 2 sockets
Aug 07 15:12:22 dist-test-slave-3kk6 krb5kdc[2893](info): commencing operation
krb5kdc: starting...
W20260807 15:12:23.779604 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:12:23 dist-test-slave-3kk6 krb5kdc[2893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:12:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:12:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:12:23.917603 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:37865
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:34303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:37865,127.15.226.125:32803,127.15.226.124:33239
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:24.037165 2916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:24.037528 2916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:24.037575 2916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:24.042367 2916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:12:24.042465 2916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:24.042487 2916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:24.042505 2916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:12:24.042560 2916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:12:24.048161 2916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:34303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:37865,127.15.226.125:32803,127.15.226.124:33239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:37865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:24.049357 2916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:24.050529 2916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:24.059643 2926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:24.059644 2923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:24.059926 2916 server_base.cc:1034] running on GCE node
W20260807 15:12:24.059643 2924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:24.060817 2916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:24.062036 2916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:24.063237 2916 hybrid_clock.cc:648] HybridClock initialized: now 1786115544063215 us; error 36 us; skew 500 ppm
Aug 07 15:12:24 dist-test-slave-3kk6 krb5kdc[2893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115544, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:24.066365 2916 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:12:24.067404 2916 webserver.cc:460] Webserver started at http://127.15.226.126:43393/ using document root <none> and password file <none>
I20260807 15:12:24.068056 2916 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:24.068131 2916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:24.068341 2916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:24.070578 2916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "c35d1a3656144668bac24591aca22652"
format_stamp: "Formatted at 2026-08-07 15:12:24 on dist-test-slave-3kk6"
server_key: "c92044771485211a8c2234b498fdc18a"
server_key_iv: "0ba5c081cc412eec5c689fffff8eb593"
server_key_version: "encryptionkey@0"
I20260807 15:12:24.071235 2916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "c35d1a3656144668bac24591aca22652"
format_stamp: "Formatted at 2026-08-07 15:12:24 on dist-test-slave-3kk6"
server_key: "c92044771485211a8c2234b498fdc18a"
server_key_iv: "0ba5c081cc412eec5c689fffff8eb593"
server_key_version: "encryptionkey@0"
I20260807 15:12:24.075301 2916 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:12:24.077967 2937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:24.079375 2916 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:12:24.079491 2916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "c35d1a3656144668bac24591aca22652"
format_stamp: "Formatted at 2026-08-07 15:12:24 on dist-test-slave-3kk6"
server_key: "c92044771485211a8c2234b498fdc18a"
server_key_iv: "0ba5c081cc412eec5c689fffff8eb593"
server_key_version: "encryptionkey@0"
I20260807 15:12:24.079592 2916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:24.113184 2916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:24.116439 2916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:24.116616 2916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:24.124480 2916 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:37865
I20260807 15:12:24.124504 3015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:37865 every 8 connection(s)
I20260807 15:12:24.125558 2916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:12:24.125902 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 2916
I20260807 15:12:24.126037 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:12:24.126379 16265 external_mini_cluster.cc:1428] Setting key e30a6e5d3eaf0b30a6081e9eb2d7eba0
I20260807 15:12:24.128645 3017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:24.134006 3017 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37865 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.226.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.125@KRBTEST.COM" created.
Aug 07 15:12:24 dist-test-slave-3kk6 krb5kdc[2893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115544, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:12:24.151569 3025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.136604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42148 (local address 127.15.226.126:37865)
0807 15:12:24.137375 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:24.137388 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:24.137526 (+ 138us) server_negotiation.cc:407] Connection header received
0807 15:12:24.138697 (+ 1171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:24.138709 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:24.138903 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:24.139150 (+ 247us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:24.140125 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.141027 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.142597 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.143069 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.143219 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:24.146307 (+ 3088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:24.146313 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:24.146330 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:24.146352 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:24.148193 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.148717 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.148730 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.148732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.148845 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.149173 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.149179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.149180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.149590 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:24.149734 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:24.150126 (+ 392us) server_negotiation.cc:299] Negotiation successful
0807 15:12:24.150443 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":649,"spinlock_wait_cycles":72064,"thread_start_us":187,"threads_started":1}
I20260807 15:12:24.151582 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.136604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:37865 (local address 127.0.0.1:42148)
0807 15:12:24.137296 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.137363 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:24.138293 (+ 930us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:24.139160 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:24.139166 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:24.139646 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:24.139953 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:24.139970 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.141166 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.141170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:24.142438 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:24.142455 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.143270 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.143278 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:24.143462 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:24.144208 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:24.144235 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:24.146122 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:24.148328 (+ 2206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:24.148337 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:24.148348 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:24.148598 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:24.148951 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:24.148954 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:24.148956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:24.149077 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:24.149718 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:24.149725 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:24.149874 (+ 149us) client_negotiation.cc:769] Sending connection context
0807 15:12:24.150049 (+ 175us) client_negotiation.cc:240] Negotiation successful
0807 15:12:24.150497 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":113,"thread_start_us":187,"threads_started":1}
I20260807 15:12:24.154398 3017 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:12:24.155301 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.154967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:32803 (local address 127.0.0.1:41208)
0807 15:12:24.155125 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.155223 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.125:32803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260807 15:12:24.155535 2942 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.226.125:32803: connect: Connection refused (error 111)
I20260807 15:12:24.156540 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.156268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:32803 (local address 127.0.0.1:41222)
0807 15:12:24.156429 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.156481 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.125:32803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 15:12:24.156761 3017 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.125:32803: Network error: Client connection negotiation failed: client connection to 127.15.226.125:32803: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.125.keytab.
Entry for principal kudu/127.15.226.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.125.keytab.
I20260807 15:12:24.158773 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.125:32803
--webserver_interface=127.15.226.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.125.keytab
--principal=kudu/127.15.226.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:34303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:37865,127.15.226.125:32803,127.15.226.124:33239
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:12:24.206063 3017 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } attempt: 1
I20260807 15:12:24.207135 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.206799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:32803 (local address 127.0.0.1:41230)
0807 15:12:24.206969 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.207048 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.125:32803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 15:12:24.208007 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.207761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:32803 (local address 127.0.0.1:41232)
0807 15:12:24.207887 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.207944 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.125:32803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 15:12:24.208145 3017 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.125:32803: Network error: Client connection negotiation failed: client connection to 127.15.226.125:32803: connect: Connection refused (error 111)
I20260807 15:12:24.276305 3017 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } attempt: 2
I20260807 15:12:24.277249 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.276935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:32803 (local address 127.0.0.1:41236)
0807 15:12:24.277096 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.277169 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.125:32803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 15:12:24.278052 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.277808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:32803 (local address 127.0.0.1:41242)
0807 15:12:24.277947 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.277994 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.125:32803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260807 15:12:24.278189 3017 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.125:32803: Network error: Client connection negotiation failed: client connection to 127.15.226.125:32803: connect: Connection refused (error 111)
W20260807 15:12:24.279515 3033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:24.279819 3033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:24.279922 3033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:24.284219 3033 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:12:24.284307 3033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:24.284344 3033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:24.284399 3033 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:12:24.284431 3033 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:12:24.290184 3033 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:34303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:37865,127.15.226.125:32803,127.15.226.124:33239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.125.keytab
--principal=kudu/127.15.226.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.125:32803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.226.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:24.291458 3033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:24.292563 3033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:24.302129 3039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:24.302129 3042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:24.302147 3040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:24.302541 3033 server_base.cc:1034] running on GCE node
I20260807 15:12:24.303164 3033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:24.304342 3033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:24.305513 3033 hybrid_clock.cc:648] HybridClock initialized: now 1786115544305494 us; error 33 us; skew 500 ppm
Aug 07 15:12:24 dist-test-slave-3kk6 krb5kdc[2893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115544, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:24.308588 3033 init.cc:377] Logged in from keytab as kudu/127.15.226.125@KRBTEST.COM (short username kudu)
I20260807 15:12:24.309624 3033 webserver.cc:460] Webserver started at http://127.15.226.125:33527/ using document root <none> and password file <none>
I20260807 15:12:24.310227 3033 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:24.310328 3033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:24.310566 3033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:24.312800 3033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-1/data/instance:
uuid: "359504a3d5f340508747bfb6066b6e4e"
format_stamp: "Formatted at 2026-08-07 15:12:24 on dist-test-slave-3kk6"
server_key: "f5da26694a9622ccf2c3e3e1edb41469"
server_key_iv: "2d19ab3dd5e165828aa6f3f26b7bcfef"
server_key_version: "encryptionkey@0"
I20260807 15:12:24.313432 3033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-1/wal/instance:
uuid: "359504a3d5f340508747bfb6066b6e4e"
format_stamp: "Formatted at 2026-08-07 15:12:24 on dist-test-slave-3kk6"
server_key: "f5da26694a9622ccf2c3e3e1edb41469"
server_key_iv: "2d19ab3dd5e165828aa6f3f26b7bcfef"
server_key_version: "encryptionkey@0"
I20260807 15:12:24.317517 3033 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:12:24.320216 3053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:24.321493 3033 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:12:24.321656 3033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-1/wal
uuid: "359504a3d5f340508747bfb6066b6e4e"
format_stamp: "Formatted at 2026-08-07 15:12:24 on dist-test-slave-3kk6"
server_key: "f5da26694a9622ccf2c3e3e1edb41469"
server_key_iv: "2d19ab3dd5e165828aa6f3f26b7bcfef"
server_key_version: "encryptionkey@0"
I20260807 15:12:24.321789 3033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:24.333185 3033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:24.336465 3033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:24.336696 3033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:24.344112 3033 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.125:32803
I20260807 15:12:24.344435 3137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.125:32803 every 8 connection(s)
I20260807 15:12:24.345194 3033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-1/data/info.pb
I20260807 15:12:24.347033 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 3033
I20260807 15:12:24.347203 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-1/wal/instance
I20260807 15:12:24.347556 16265 external_mini_cluster.cc:1428] Setting key dff00c4360bc08e6d8e9c9cbc79e3e43
I20260807 15:12:24.348480 3138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:24.353210 3138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37865 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.226.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.124@KRBTEST.COM" created.
Aug 07 15:12:24 dist-test-slave-3kk6 krb5kdc[2893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115544, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.125@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:12:24.369212 3025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.355385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42154 (local address 127.15.226.126:37865)
0807 15:12:24.355543 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:24.355547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:24.356761 (+ 1214us) server_negotiation.cc:407] Connection header received
0807 15:12:24.357828 (+ 1067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:24.357831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:24.357900 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:24.358020 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:24.359680 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.360395 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.361605 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.361869 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.361990 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:24.365118 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:24.365122 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:24.365124 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:24.365153 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:24.366844 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.367443 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.367446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.367448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.367534 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.367956 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.367959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.367961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.368169 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:24.368287 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:24.368875 (+ 588us) server_negotiation.cc:299] Negotiation successful
0807 15:12:24.369025 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 15:12:24.369431 3017 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } attempt: 3
I20260807 15:12:24.370330 3143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.355917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:37865 (local address 127.0.0.1:42154)
0807 15:12:24.356568 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.356615 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:24.357588 (+ 973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:24.358310 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:24.358322 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:24.358793 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:24.359506 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:24.359525 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.360540 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.360544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:24.361461 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:24.361472 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.362052 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.362056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:24.362316 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:24.363083 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:24.363111 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:24.364908 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:24.366984 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:24.366991 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:24.367010 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:24.367309 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:24.367637 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:24.367641 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:24.367642 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:24.367826 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:24.368289 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:24.368299 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:24.368593 (+ 294us) client_negotiation.cc:769] Sending connection context
0807 15:12:24.368818 (+ 225us) client_negotiation.cc:240] Negotiation successful
0807 15:12:24.369184 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":162,"threads_started":1}
I20260807 15:12:24.372051 3138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:12:24 dist-test-slave-3kk6 krb5kdc[2893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115544, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for kudu/127.15.226.125@KRBTEST.COM
Aug 07 15:12:24 dist-test-slave-3kk6 krb5kdc[2893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115544, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.125@KRBTEST.COM for kudu/127.15.226.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.124.keytab.
Entry for principal kudu/127.15.226.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.124.keytab.
I20260807 15:12:24.382169 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.124:33239
--webserver_interface=127.15.226.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.124.keytab
--principal=kudu/127.15.226.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:34303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:37865,127.15.226.125:32803,127.15.226.124:33239
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:12:24.383172 3143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.372659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:32803 (local address 127.0.0.1:41258)
0807 15:12:24.372789 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.372804 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:24.372885 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:24.373370 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:24.373374 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:24.373556 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:24.373671 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:24.373678 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.374522 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.374525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:24.375225 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:24.375235 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.375880 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.375884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:24.376002 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:24.376484 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:24.376505 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:24.378078 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:24.381052 (+ 2974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:24.381055 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:24.381057 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:24.381317 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:24.381734 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:24.381736 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:24.381738 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:24.381846 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:24.382430 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:24.382435 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:24.382543 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 15:12:24.382704 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 15:12:24.382890 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 15:12:24.383438 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.369920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:32803 (local address 127.0.0.1:41246)
0807 15:12:24.370083 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.370100 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:24.370195 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:24.370946 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:24.370950 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:24.371130 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:24.371272 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:24.371287 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.372585 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.372589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:24.374188 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:24.374200 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.375060 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.375064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:24.375185 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:24.375847 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:24.375867 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:24.377520 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:24.381069 (+ 3549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:24.381072 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:24.381074 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:24.381346 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:24.381690 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:24.381693 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:24.381695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:24.381790 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:24.382448 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:24.382451 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:24.382641 (+ 190us) client_negotiation.cc:769] Sending connection context
0807 15:12:24.382796 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 15:12:24.382961 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 15:12:24.383203 3146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.372707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41258 (local address 127.15.226.125:32803)
0807 15:12:24.373111 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:24.373125 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:24.373141 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:12:24.373179 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:24.373182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:24.373229 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:24.373302 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:24.373791 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.374402 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.375391 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.375753 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.375856 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:24.378258 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:24.378265 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:24.378269 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:24.378297 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:24.380916 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.381463 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.381472 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.381473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.381622 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.381951 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.381953 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.381955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.382233 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:24.382381 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:24.382832 (+ 451us) server_negotiation.cc:299] Negotiation successful
0807 15:12:24.382999 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":189,"threads_started":1}
I20260807 15:12:24.383852 3145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.370109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41246 (local address 127.15.226.125:32803)
0807 15:12:24.370474 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:24.370487 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:24.370511 (+ 24us) server_negotiation.cc:407] Connection header received
0807 15:12:24.370557 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:24.370562 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:24.370796 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:24.370939 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:24.371412 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.372405 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.374311 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.374935 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.375026 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:24.377701 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:24.377718 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:24.377738 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:24.377770 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:24.380916 (+ 3146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.381470 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.381473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.381474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.381574 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.381901 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.381905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.381906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.382228 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:24.382381 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:24.383205 (+ 824us) server_negotiation.cc:299] Negotiation successful
0807 15:12:24.383352 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"spinlock_wait_cycles":1792,"thread_start_us":119,"threads_started":1}
I20260807 15:12:24.385618 3017 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:12:24.385825 3138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:12:24.386628 3143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.386339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:33239 (local address 127.0.0.1:37692)
0807 15:12:24.386485 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.386572 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:33239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 15:12:24.386754 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.386371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:33239 (local address 127.0.0.1:37686)
0807 15:12:24.386579 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.386645 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:33239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260807 15:12:24.387001 3056 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.226.124:33239: connect: Connection refused (error 111)
I20260807 15:12:24.387812 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.387290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:33239 (local address 127.0.0.1:37698)
0807 15:12:24.387655 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.387717 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:33239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":103,"threads_started":1}
W20260807 15:12:24.387945 3017 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.124:33239: Network error: Client connection negotiation failed: client connection to 127.15.226.124:33239: connect: Connection refused (error 111)
I20260807 15:12:24.388188 3143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.387734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:33239 (local address 127.0.0.1:37702)
0807 15:12:24.388074 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.388129 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:33239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 15:12:24.388373 3138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.124:33239: Network error: Client connection negotiation failed: client connection to 127.15.226.124:33239: connect: Connection refused (error 111)
I20260807 15:12:24.419145 3017 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } attempt: 1
I20260807 15:12:24.420274 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.419895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:33239 (local address 127.0.0.1:37716)
0807 15:12:24.420088 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.420171 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:33239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 15:12:24.421115 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.420846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:33239 (local address 127.0.0.1:37724)
0807 15:12:24.420971 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.421030 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:33239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 15:12:24.421280 3017 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.124:33239: Network error: Client connection negotiation failed: client connection to 127.15.226.124:33239: connect: Connection refused (error 111)
I20260807 15:12:24.437633 3138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } attempt: 1
I20260807 15:12:24.438761 3143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.438346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:33239 (local address 127.0.0.1:37728)
0807 15:12:24.438544 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.438631 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:33239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 15:12:24.439608 3143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.439352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:33239 (local address 127.0.0.1:37730)
0807 15:12:24.439480 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.439540 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:33239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 15:12:24.439796 3138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.124:33239: Network error: Client connection negotiation failed: client connection to 127.15.226.124:33239: connect: Connection refused (error 111)
I20260807 15:12:24.496505 3017 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } attempt: 2
I20260807 15:12:24.497534 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.497129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:33239 (local address 127.0.0.1:37746)
0807 15:12:24.497342 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.497428 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:33239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 15:12:24.498404 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.498138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:33239 (local address 127.0.0.1:37748)
0807 15:12:24.498269 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.498335 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:33239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 15:12:24.498588 3017 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.124:33239: Network error: Client connection negotiation failed: client connection to 127.15.226.124:33239: connect: Connection refused (error 111)
W20260807 15:12:24.505317 3147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:24.505669 3147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:24.505779 3147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 15:12:24.507977 3138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } attempt: 2
I20260807 15:12:24.508863 3143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.508527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:33239 (local address 127.0.0.1:37750)
0807 15:12:24.508695 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.508769 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:33239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 15:12:24.509619 3143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.509401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:33239 (local address 127.0.0.1:37764)
0807 15:12:24.509508 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.509560 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:33239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 15:12:24.509750 3138 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.124:33239: Network error: Client connection negotiation failed: client connection to 127.15.226.124:33239: connect: Connection refused (error 111)
W20260807 15:12:24.510776 3147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:12:24.510835 3147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:24.510855 3147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:24.510906 3147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:12:24.510941 3147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:12:24.516618 3147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:34303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:37865,127.15.226.125:32803,127.15.226.124:33239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.124.keytab
--principal=kudu/127.15.226.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.124:33239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.226.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:24.517817 3147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:24.518987 3147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:24.528194 3167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:24.528232 3163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:24.528441 3147 server_base.cc:1034] running on GCE node
W20260807 15:12:24.528194 3164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:24.529178 3147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:24.530344 3147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:24.531517 3147 hybrid_clock.cc:648] HybridClock initialized: now 1786115544531478 us; error 59 us; skew 500 ppm
Aug 07 15:12:24 dist-test-slave-3kk6 krb5kdc[2893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115544, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:24.534453 3147 init.cc:377] Logged in from keytab as kudu/127.15.226.124@KRBTEST.COM (short username kudu)
I20260807 15:12:24.535506 3147 webserver.cc:460] Webserver started at http://127.15.226.124:37339/ using document root <none> and password file <none>
I20260807 15:12:24.536134 3147 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:24.536238 3147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:24.536499 3147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:24.538825 3147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-2/data/instance:
uuid: "820b03e8358c483e901d2c92c2ba7f26"
format_stamp: "Formatted at 2026-08-07 15:12:24 on dist-test-slave-3kk6"
server_key: "666530ba2e18229b878e317baaa9322a"
server_key_iv: "5a6a6d3fd9a88a524a53d997d9f67103"
server_key_version: "encryptionkey@0"
I20260807 15:12:24.539494 3147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-2/wal/instance:
uuid: "820b03e8358c483e901d2c92c2ba7f26"
format_stamp: "Formatted at 2026-08-07 15:12:24 on dist-test-slave-3kk6"
server_key: "666530ba2e18229b878e317baaa9322a"
server_key_iv: "5a6a6d3fd9a88a524a53d997d9f67103"
server_key_version: "encryptionkey@0"
I20260807 15:12:24.543833 3147 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 15:12:24.546792 3180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:24.548348 3147 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:12:24.548533 3147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-2/wal
uuid: "820b03e8358c483e901d2c92c2ba7f26"
format_stamp: "Formatted at 2026-08-07 15:12:24 on dist-test-slave-3kk6"
server_key: "666530ba2e18229b878e317baaa9322a"
server_key_iv: "5a6a6d3fd9a88a524a53d997d9f67103"
server_key_version: "encryptionkey@0"
I20260807 15:12:24.548687 3147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:24.569223 3147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:24.573303 3147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:24.573550 3147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:24.581329 3147 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.124:33239
I20260807 15:12:24.581384 3248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.124:33239 every 8 connection(s)
I20260807 15:12:24.582439 3147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-2/data/info.pb
I20260807 15:12:24.582744 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 3147
I20260807 15:12:24.582911 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786115365131938-16265-0/minicluster-data/master-2/wal/instance
I20260807 15:12:24.583289 16265 external_mini_cluster.cc:1428] Setting key 4c4f1a90043208b1ada41b5180831800
I20260807 15:12:24.585922 3249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:24.591315 3249 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37865 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:12:24 dist-test-slave-3kk6 krb5kdc[2893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.124@KRBTEST.COM
I20260807 15:12:24.597854 3017 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } attempt: 3
Aug 07 15:12:24 dist-test-slave-3kk6 krb5kdc[2893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115544, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.124@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:12:24.600596 3254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.584886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37772 (local address 127.15.226.124:33239)
0807 15:12:24.585324 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:24.585341 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:24.585378 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:12:24.586113 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:24.586139 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:24.586493 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:24.586891 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:24.588012 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.589286 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.590551 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.591132 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.591264 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:24.594682 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:24.594693 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:24.594716 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:24.594752 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:24.596555 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.597045 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.597052 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.597058 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.597204 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.597516 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.597518 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.597520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.598052 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:24.598913 (+ 861us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:24.599090 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 15:12:24.599420 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":153,"threads_started":1}
I20260807 15:12:24.601671 3138 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } attempt: 3
Aug 07 15:12:24 dist-test-slave-3kk6 krb5kdc[2893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115544, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for kudu/127.15.226.124@KRBTEST.COM
I20260807 15:12:24.605505 3025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.593908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42160 (local address 127.15.226.126:37865)
0807 15:12:24.594060 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:24.594065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:24.594455 (+ 390us) server_negotiation.cc:407] Connection header received
0807 15:12:24.594740 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:24.594743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:24.594801 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:24.594881 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:24.595841 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.596344 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.597443 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.597697 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.597808 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:24.600326 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:24.600330 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:24.600333 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:24.600359 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:24.602248 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.603699 (+ 1451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.603702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.603704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.603787 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.604138 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.604145 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.604147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.604396 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:24.604483 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:24.605006 (+ 523us) server_negotiation.cc:299] Negotiation successful
0807 15:12:24.605341 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:12:24.606020 3256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.593738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:37865 (local address 127.0.0.1:42160)
0807 15:12:24.594318 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.594361 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:24.594517 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:24.594952 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:24.594960 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:24.595459 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:24.595658 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:24.595667 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.596467 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.596473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:24.597316 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:24.597324 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.597812 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.597816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:24.597921 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:24.598480 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:24.598495 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:24.600178 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:24.602378 (+ 2200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:24.602386 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:24.602391 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:24.602713 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:24.603908 (+ 1195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:24.603912 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:24.603914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:24.604011 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:24.604622 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:24.604628 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:24.604759 (+ 131us) client_negotiation.cc:769] Sending connection context
0807 15:12:24.604926 (+ 167us) client_negotiation.cc:240] Negotiation successful
0807 15:12:24.605173 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":195,"threads_started":1}
I20260807 15:12:24.607313 3249 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:12:24 dist-test-slave-3kk6 krb5kdc[2893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115544, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.125@KRBTEST.COM for kudu/127.15.226.124@KRBTEST.COM
I20260807 15:12:24.611236 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.598379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:33239 (local address 127.0.0.1:37788)
0807 15:12:24.598559 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.598571 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:24.598631 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:24.599485 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:24.599488 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:24.599669 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:24.599770 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:24.599777 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.600829 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.600832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:24.601563 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:24.601576 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.603007 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.603011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:24.603114 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:24.603586 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:24.603601 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:24.605213 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:24.609603 (+ 4390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:24.609606 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:24.609608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:24.609829 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:24.610138 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:24.610141 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:24.610143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:24.610240 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:24.610765 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:24.610768 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:24.610862 (+ 94us) client_negotiation.cc:769] Sending connection context
0807 15:12:24.610942 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 15:12:24.611082 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
Aug 07 15:12:24 dist-test-slave-3kk6 krb5kdc[2893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:12:24.612174 3258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.598831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37788 (local address 127.15.226.124:33239)
0807 15:12:24.599181 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:24.599209 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:24.599227 (+ 18us) server_negotiation.cc:407] Connection header received
0807 15:12:24.599263 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:24.599266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:24.599328 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:24.599434 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:24.599916 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.600685 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.602016 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.602343 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.602980 (+ 637us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:24.605962 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:24.605966 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:24.605969 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:24.606015 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:24.609066 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.609937 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.609939 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.609941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.610031 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.610341 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.610346 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.610348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.610619 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:24.611753 (+ 1134us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:24.611840 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 15:12:24.611976 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":289,"thread_start_us":147,"threads_started":1}
I20260807 15:12:24.613214 3143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.602178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:33239 (local address 127.0.0.1:37794)
0807 15:12:24.602468 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.602485 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:24.602571 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:24.602896 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:24.602899 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:24.603120 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:24.603255 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:24.603268 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.604325 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.604331 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:24.605101 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:24.605116 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.605799 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.605806 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:24.605909 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:24.606415 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:24.606436 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:24.608017 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:24.611280 (+ 3263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:24.611284 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:24.611289 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:24.611581 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:24.611979 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:24.611982 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:24.611984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:24.612086 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:24.612588 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:24.612593 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:24.612728 (+ 135us) client_negotiation.cc:769] Sending connection context
0807 15:12:24.612851 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 15:12:24.613004 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197}
I20260807 15:12:24.613217 3254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.602416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37794 (local address 127.15.226.124:33239)
0807 15:12:24.602604 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:24.602609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:24.602632 (+ 23us) server_negotiation.cc:407] Connection header received
0807 15:12:24.602696 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:24.602698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:24.602743 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:24.602852 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:24.603403 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.604159 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.605280 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.605658 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.606152 (+ 494us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:24.608186 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:24.608190 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:24.608193 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:24.608232 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:24.611147 (+ 2915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.611746 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.611749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.611750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.611861 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.612223 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.612229 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.612233 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.612476 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:24.612594 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:24.612919 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 15:12:24.613050 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"spinlock_wait_cycles":2944}
I20260807 15:12:24.616170 3260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.604219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42170 (local address 127.15.226.126:37865)
0807 15:12:24.605244 (+ 1025us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:24.605262 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:24.605280 (+ 18us) server_negotiation.cc:407] Connection header received
0807 15:12:24.605326 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:24.605330 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:24.605377 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:24.606362 (+ 985us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:24.606469 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.607258 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.608362 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.608771 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.609265 (+ 494us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:24.611974 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:24.611978 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:24.611981 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:24.612005 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:24.613840 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.614790 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.614793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.614794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.614883 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.615216 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.615219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.615221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.615444 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:24.615656 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:24.615886 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 15:12:24.616004 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":950,"thread_start_us":99,"threads_started":1}
I20260807 15:12:24.616858 3017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c35d1a3656144668bac24591aca22652: Bootstrap starting.
I20260807 15:12:24.616852 3138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 359504a3d5f340508747bfb6066b6e4e: Bootstrap starting.
I20260807 15:12:24.619781 3017 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c35d1a3656144668bac24591aca22652: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:24.619776 3138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 359504a3d5f340508747bfb6066b6e4e: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:24.620544 3017 log.cc:824] T 00000000000000000000000000000000 P c35d1a3656144668bac24591aca22652: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:24.620700 3138 log.cc:824] T 00000000000000000000000000000000 P 359504a3d5f340508747bfb6066b6e4e: Log is configured to *not* fsync() on all Append() calls
Aug 07 15:12:24 dist-test-slave-3kk6 krb5kdc[2893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115544, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.124@KRBTEST.COM for kudu/127.15.226.125@KRBTEST.COM
I20260807 15:12:24.623265 3138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 359504a3d5f340508747bfb6066b6e4e: No bootstrap required, opened a new log
I20260807 15:12:24.623265 3017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c35d1a3656144668bac24591aca22652: No bootstrap required, opened a new log
Aug 07 15:12:24 dist-test-slave-3kk6 krb5kdc[2893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.125@KRBTEST.COM
I20260807 15:12:24.626434 3256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.607833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:32803 (local address 127.0.0.1:41272)
0807 15:12:24.608460 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.608474 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:24.608557 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:24.608966 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:24.608970 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:24.609157 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:24.609285 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:24.609297 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.610372 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.610375 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:24.611247 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:24.611262 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.611997 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.612001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:24.612088 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:24.613599 (+ 1511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:24.613617 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:24.622369 (+ 8752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:24.624529 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:24.624532 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:24.624534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:24.624802 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:24.625120 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:24.625147 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:24.625149 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:24.625236 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:24.625823 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:24.625826 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:24.625939 (+ 113us) client_negotiation.cc:769] Sending connection context
0807 15:12:24.626049 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 15:12:24.626209 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"mutex_wait_us":20,"thread_start_us":123,"threads_started":1}
I20260807 15:12:24.627137 3145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.608212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41272 (local address 127.15.226.125:32803)
0807 15:12:24.608351 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:24.608354 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:24.608530 (+ 176us) server_negotiation.cc:407] Connection header received
0807 15:12:24.608677 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:24.608681 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:24.608747 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:24.608832 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:24.609439 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.610252 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.611401 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.611779 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.611855 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:24.622554 (+ 10699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:24.622558 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:24.622560 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:24.622585 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:24.624379 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.624922 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.624926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.624928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.625017 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.625394 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.625398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.625400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.625672 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:24.625771 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:24.626847 (+ 1076us) server_negotiation.cc:299] Negotiation successful
0807 15:12:24.626983 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:12:24.627923 3138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 359504a3d5f340508747bfb6066b6e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c35d1a3656144668bac24591aca22652" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37865 } } peers { permanent_uuid: "359504a3d5f340508747bfb6066b6e4e" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } } peers { permanent_uuid: "820b03e8358c483e901d2c92c2ba7f26" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } }
I20260807 15:12:24.628185 3138 raft_consensus.cc:374] T 00000000000000000000000000000000 P 359504a3d5f340508747bfb6066b6e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:24.628248 3138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 359504a3d5f340508747bfb6066b6e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 359504a3d5f340508747bfb6066b6e4e, State: Initialized, Role: FOLLOWER
I20260807 15:12:24.628191 3017 raft_consensus.cc:348] T 00000000000000000000000000000000 P c35d1a3656144668bac24591aca22652 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c35d1a3656144668bac24591aca22652" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37865 } } peers { permanent_uuid: "359504a3d5f340508747bfb6066b6e4e" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } } peers { permanent_uuid: "820b03e8358c483e901d2c92c2ba7f26" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } }
I20260807 15:12:24.628336 3249 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:12:24.628414 3017 raft_consensus.cc:374] T 00000000000000000000000000000000 P c35d1a3656144668bac24591aca22652 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:24.628465 3017 raft_consensus.cc:729] T 00000000000000000000000000000000 P c35d1a3656144668bac24591aca22652 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c35d1a3656144668bac24591aca22652, State: Initialized, Role: FOLLOWER
I20260807 15:12:24.628794 3138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 359504a3d5f340508747bfb6066b6e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c35d1a3656144668bac24591aca22652" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37865 } } peers { permanent_uuid: "359504a3d5f340508747bfb6066b6e4e" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } } peers { permanent_uuid: "820b03e8358c483e901d2c92c2ba7f26" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } }
I20260807 15:12:24.628942 3017 consensus_queue.cc:260] T 00000000000000000000000000000000 P c35d1a3656144668bac24591aca22652 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c35d1a3656144668bac24591aca22652" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37865 } } peers { permanent_uuid: "359504a3d5f340508747bfb6066b6e4e" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } } peers { permanent_uuid: "820b03e8358c483e901d2c92c2ba7f26" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } }
I20260807 15:12:24.629626 3146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.617335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41274 (local address 127.15.226.125:32803)
0807 15:12:24.617461 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:24.617465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:24.617503 (+ 38us) server_negotiation.cc:407] Connection header received
0807 15:12:24.617641 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:24.617644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:24.617690 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:24.617772 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:24.618546 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.619344 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.620535 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.620875 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.620993 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:24.624844 (+ 3851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:24.624847 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:24.624850 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:24.624876 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:24.626941 (+ 2065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.627682 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.627686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.627688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.627795 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.628124 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.628127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.628129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.628494 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:24.628680 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:24.629367 (+ 687us) server_negotiation.cc:299] Negotiation successful
0807 15:12:24.629494 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:12:24.630183 3138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 359504a3d5f340508747bfb6066b6e4e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:12:24.630313 3017 sys_catalog.cc:564] T 00000000000000000000000000000000 P c35d1a3656144668bac24591aca22652 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:12:24.630918 3267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 359504a3d5f340508747bfb6066b6e4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c35d1a3656144668bac24591aca22652" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37865 } } peers { permanent_uuid: "359504a3d5f340508747bfb6066b6e4e" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } } peers { permanent_uuid: "820b03e8358c483e901d2c92c2ba7f26" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } } }
I20260807 15:12:24.631069 3267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 359504a3d5f340508747bfb6066b6e4e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:12:24.633692 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
Aug 07 15:12:24 dist-test-slave-3kk6 krb5kdc[2893](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115544, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.124@KRBTEST.COM for kudu/127.15.226.124@KRBTEST.COM
W20260807 15:12:24.636372 3289 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 359504a3d5f340508747bfb6066b6e4e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:12:24.636466 3289 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:12:24.634123 3033 master_runner.cc:186] Error getting master registration for 127.15.226.126:37865: OK, instance_id { permanent_uuid: "c35d1a3656144668bac24591aca22652" instance_seqno: 1786115544122050 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:12:24.637599 3033 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:12:24.640460 3256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.628801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:33239 (local address 127.0.0.1:37806)
0807 15:12:24.629119 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:24.629135 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:24.629217 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:24.630807 (+ 1590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:24.630810 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:24.630991 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:24.631104 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:24.631115 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.632254 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.632257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:24.633064 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:24.633076 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.633663 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.633667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:24.633766 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:24.634408 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:24.634424 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:24.636183 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:24.638776 (+ 2593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:24.638783 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:24.638789 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:24.639063 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:24.639387 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:24.639390 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:24.639392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:24.639486 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:24.639996 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:24.639999 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:24.640090 (+ 91us) client_negotiation.cc:769] Sending connection context
0807 15:12:24.640172 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 15:12:24.640305 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233}
I20260807 15:12:24.641266 3258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.628999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37806 (local address 127.15.226.124:33239)
0807 15:12:24.630554 (+ 1555us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:24.630559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:24.630573 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:12:24.630608 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:24.630611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:24.630678 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:24.631307 (+ 629us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:24.631398 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.632136 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.633198 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.633539 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.636328 (+ 2789us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:24.636438 (+ 110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:24.636442 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:24.636444 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:24.636571 (+ 127us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:24.638616 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.639178 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.639181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.639182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.639266 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.639599 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.639603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.639605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.639883 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:24.640902 (+ 1019us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:24.640980 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 15:12:24.641099 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1471}
W20260807 15:12:24.642839 3319 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c35d1a3656144668bac24591aca22652: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:12:24.642920 3319 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:12:24.643867 3268 sys_catalog.cc:455] T 00000000000000000000000000000000 P c35d1a3656144668bac24591aca22652 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c35d1a3656144668bac24591aca22652" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37865 } } peers { permanent_uuid: "359504a3d5f340508747bfb6066b6e4e" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } } peers { permanent_uuid: "820b03e8358c483e901d2c92c2ba7f26" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } } }
I20260807 15:12:24.643991 3268 sys_catalog.cc:458] T 00000000000000000000000000000000 P c35d1a3656144668bac24591aca22652 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:12:24.645383 3249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26: Bootstrap starting.
I20260807 15:12:24.643000 2916 master_runner.cc:186] Error getting master registration for 127.15.226.124:33239: OK, instance_id { permanent_uuid: "820b03e8358c483e901d2c92c2ba7f26" instance_seqno: 1786115544579060 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:12:24.647017 2916 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:12:24.648487 3249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:24.649014 3254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.637692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37820 (local address 127.15.226.124:33239)
0807 15:12:24.637820 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:24.637824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:24.639604 (+ 1780us) server_negotiation.cc:407] Connection header received
0807 15:12:24.639802 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:24.639805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:24.639854 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:24.639907 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:24.641097 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.641827 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.643204 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.643552 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.643627 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:24.645438 (+ 1811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:24.645443 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:24.645447 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:24.645471 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:24.647216 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.647682 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.647685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.647687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.647779 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.648083 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.648086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.648088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.648279 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:24.648446 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:24.648745 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 15:12:24.648865 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 15:12:24.649482 3249 log.cc:824] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:24.657649 3249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26: No bootstrap required, opened a new log
I20260807 15:12:24.660545 3260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.638417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42172 (local address 127.15.226.126:37865)
0807 15:12:24.638542 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:24.638547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:24.649119 (+ 10572us) server_negotiation.cc:407] Connection header received
0807 15:12:24.649322 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:24.649326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:24.649381 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:24.649440 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:24.650367 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.651278 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.652370 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.652729 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.652823 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:24.655007 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:24.655012 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:24.655016 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:24.655051 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:24.657511 (+ 2460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.658966 (+ 1455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.658970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.658972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.659071 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.659397 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.659401 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.659402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.659642 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:24.659777 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:24.660203 (+ 426us) server_negotiation.cc:299] Negotiation successful
0807 15:12:24.660369 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:12:24.662474 3146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:24.639190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41290 (local address 127.15.226.125:32803)
0807 15:12:24.639324 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:24.639328 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:24.649426 (+ 10098us) server_negotiation.cc:407] Connection header received
0807 15:12:24.649479 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:24.649482 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:24.649536 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:24.649614 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:24.651272 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.652072 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.653218 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:24.653626 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:24.653718 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:24.655571 (+ 1853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:24.655578 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:24.655584 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:24.655619 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:24.658330 (+ 2711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.658985 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.658989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.658990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.659071 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:24.661073 (+ 2002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:24.661077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:24.661081 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:24.661345 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:24.661474 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:24.662174 (+ 700us) server_negotiation.cc:299] Negotiation successful
0807 15:12:24.662325 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:12:24.665251 3249 raft_consensus.cc:348] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c35d1a3656144668bac24591aca22652" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37865 } } peers { permanent_uuid: "359504a3d5f340508747bfb6066b6e4e" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } } peers { permanent_uuid: "820b03e8358c483e901d2c92c2ba7f26" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } }
I20260807 15:12:24.665532 3249 raft_consensus.cc:374] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:24.665596 3249 raft_consensus.cc:729] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 820b03e8358c483e901d2c92c2ba7f26, State: Initialized, Role: FOLLOWER
I20260807 15:12:24.666096 3249 consensus_queue.cc:260] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c35d1a3656144668bac24591aca22652" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37865 } } peers { permanent_uuid: "359504a3d5f340508747bfb6066b6e4e" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } } peers { permanent_uuid: "820b03e8358c483e901d2c92c2ba7f26" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } }
I20260807 15:12:24.666684 3325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c35d1a3656144668bac24591aca22652" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37865 } } peers { permanent_uuid: "359504a3d5f340508747bfb6066b6e4e" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } } peers { permanent_uuid: "820b03e8358c483e901d2c92c2ba7f26" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } } }
I20260807 15:12:24.666945 3325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:12:24.667629 3249 sys_catalog.cc:564] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 15:12:24.674794 3338 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:12:24.674993 3338 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:12:24.677497 3147 master_runner.cc:263] No leader master found from master 820b03e8358c483e901d2c92c2ba7f26
I20260807 15:12:24.677582 3147 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:12:24.768508 3325 raft_consensus.cc:482] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:12:24.768766 3325 raft_consensus.cc:504] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c35d1a3656144668bac24591aca22652" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37865 } } peers { permanent_uuid: "359504a3d5f340508747bfb6066b6e4e" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } } peers { permanent_uuid: "820b03e8358c483e901d2c92c2ba7f26" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } }
I20260807 15:12:24.769939 3325 leader_election.cc:288] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c35d1a3656144668bac24591aca22652 (127.15.226.126:37865), 359504a3d5f340508747bfb6066b6e4e (127.15.226.125:32803)
I20260807 15:12:24.770728 3094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "820b03e8358c483e901d2c92c2ba7f26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "359504a3d5f340508747bfb6066b6e4e" is_pre_election: true
I20260807 15:12:24.771085 3094 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 359504a3d5f340508747bfb6066b6e4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 820b03e8358c483e901d2c92c2ba7f26 in term 0.
I20260807 15:12:24.771867 2978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "820b03e8358c483e901d2c92c2ba7f26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c35d1a3656144668bac24591aca22652" is_pre_election: true
I20260807 15:12:24.772176 2978 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c35d1a3656144668bac24591aca22652 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 820b03e8358c483e901d2c92c2ba7f26 in term 0.
I20260807 15:12:24.772738 3184 leader_election.cc:302] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 359504a3d5f340508747bfb6066b6e4e, 820b03e8358c483e901d2c92c2ba7f26; no voters:
I20260807 15:12:24.773340 3325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:12:24.773443 3325 raft_consensus.cc:482] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:12:24.773495 3325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:24.775321 3325 raft_consensus.cc:504] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c35d1a3656144668bac24591aca22652" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37865 } } peers { permanent_uuid: "359504a3d5f340508747bfb6066b6e4e" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } } peers { permanent_uuid: "820b03e8358c483e901d2c92c2ba7f26" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } }
I20260807 15:12:24.775720 3325 leader_election.cc:288] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [CANDIDATE]: Term 1 election: Requested vote from peers c35d1a3656144668bac24591aca22652 (127.15.226.126:37865), 359504a3d5f340508747bfb6066b6e4e (127.15.226.125:32803)
I20260807 15:12:24.776438 3094 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "820b03e8358c483e901d2c92c2ba7f26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "359504a3d5f340508747bfb6066b6e4e"
I20260807 15:12:24.776597 3094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 359504a3d5f340508747bfb6066b6e4e [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:24.777972 3094 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 359504a3d5f340508747bfb6066b6e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 820b03e8358c483e901d2c92c2ba7f26 in term 1.
I20260807 15:12:24.778367 2978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "820b03e8358c483e901d2c92c2ba7f26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c35d1a3656144668bac24591aca22652"
I20260807 15:12:24.778527 2978 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c35d1a3656144668bac24591aca22652 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:24.779814 2978 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c35d1a3656144668bac24591aca22652 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 820b03e8358c483e901d2c92c2ba7f26 in term 1.
I20260807 15:12:24.780196 3183 leader_election.cc:302] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 820b03e8358c483e901d2c92c2ba7f26, c35d1a3656144668bac24591aca22652; no voters:
I20260807 15:12:24.780678 3325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:12:24.780943 3325 raft_consensus.cc:686] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [term 1 LEADER]: Becoming Leader. State: Replica: 820b03e8358c483e901d2c92c2ba7f26, State: Running, Role: LEADER
I20260807 15:12:24.781325 3325 consensus_queue.cc:237] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c35d1a3656144668bac24591aca22652" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37865 } } peers { permanent_uuid: "359504a3d5f340508747bfb6066b6e4e" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } } peers { permanent_uuid: "820b03e8358c483e901d2c92c2ba7f26" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } }
I20260807 15:12:24.787184 3325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 820b03e8358c483e901d2c92c2ba7f26. Latest consensus state: current_term: 1 leader_uuid: "820b03e8358c483e901d2c92c2ba7f26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c35d1a3656144668bac24591aca22652" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37865 } } peers { permanent_uuid: "359504a3d5f340508747bfb6066b6e4e" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } } peers { permanent_uuid: "820b03e8358c483e901d2c92c2ba7f26" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } } }
I20260807 15:12:24.787351 3325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:24.787828 3346 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:12:24.789206 3346 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:12:24.795249 3094 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 359504a3d5f340508747bfb6066b6e4e [term 1 FOLLOWER]: Refusing update from remote peer 820b03e8358c483e901d2c92c2ba7f26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:12:24.796157 3341 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "359504a3d5f340508747bfb6066b6e4e" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:12:24.801105 3267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 359504a3d5f340508747bfb6066b6e4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 820b03e8358c483e901d2c92c2ba7f26. Latest consensus state: current_term: 1 leader_uuid: "820b03e8358c483e901d2c92c2ba7f26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c35d1a3656144668bac24591aca22652" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37865 } } peers { permanent_uuid: "359504a3d5f340508747bfb6066b6e4e" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } } peers { permanent_uuid: "820b03e8358c483e901d2c92c2ba7f26" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } } }
I20260807 15:12:24.801288 3267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 359504a3d5f340508747bfb6066b6e4e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:12:24.805423 2978 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c35d1a3656144668bac24591aca22652 [term 1 FOLLOWER]: Refusing update from remote peer 820b03e8358c483e901d2c92c2ba7f26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:12:24.806104 3325 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c35d1a3656144668bac24591aca22652" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:12:24.810772 3268 sys_catalog.cc:455] T 00000000000000000000000000000000 P c35d1a3656144668bac24591aca22652 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 820b03e8358c483e901d2c92c2ba7f26. Latest consensus state: current_term: 1 leader_uuid: "820b03e8358c483e901d2c92c2ba7f26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c35d1a3656144668bac24591aca22652" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37865 } } peers { permanent_uuid: "359504a3d5f340508747bfb6066b6e4e" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } } peers { permanent_uuid: "820b03e8358c483e901d2c92c2ba7f26" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } } }
I20260807 15:12:24.810938 3268 sys_catalog.cc:458] T 00000000000000000000000000000000 P c35d1a3656144668bac24591aca22652 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:12:24.818396 3341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "820b03e8358c483e901d2c92c2ba7f26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c35d1a3656144668bac24591aca22652" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37865 } } peers { permanent_uuid: "359504a3d5f340508747bfb6066b6e4e" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } } peers { permanent_uuid: "820b03e8358c483e901d2c92c2ba7f26" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } } }
I20260807 15:12:24.818598 3341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:24.823221 3268 sys_catalog.cc:455] T 00000000000000000000000000000000 P c35d1a3656144668bac24591aca22652 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "820b03e8358c483e901d2c92c2ba7f26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c35d1a3656144668bac24591aca22652" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37865 } } peers { permanent_uuid: "359504a3d5f340508747bfb6066b6e4e" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } } peers { permanent_uuid: "820b03e8358c483e901d2c92c2ba7f26" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } } }
I20260807 15:12:24.823403 3268 sys_catalog.cc:458] T 00000000000000000000000000000000 P c35d1a3656144668bac24591aca22652 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:12:24.823865 3347 mvcc.cc:205] Tried to move back new op lower bound from 7315929271472549888 to 7315929271439429632. Current Snapshot: MvccSnapshot[applied={T|T < 7315929271472549888}]
I20260807 15:12:24.825007 3354 mvcc.cc:205] Tried to move back new op lower bound from 7315929271472549888 to 7315929271439429632. Current Snapshot: MvccSnapshot[applied={T|T < 7315929271472549888}]
I20260807 15:12:24.827203 3325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "820b03e8358c483e901d2c92c2ba7f26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c35d1a3656144668bac24591aca22652" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37865 } } peers { permanent_uuid: "359504a3d5f340508747bfb6066b6e4e" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } } peers { permanent_uuid: "820b03e8358c483e901d2c92c2ba7f26" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } } }
I20260807 15:12:24.827366 3325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26 [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:24.828672 3267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 359504a3d5f340508747bfb6066b6e4e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "820b03e8358c483e901d2c92c2ba7f26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c35d1a3656144668bac24591aca22652" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 37865 } } peers { permanent_uuid: "359504a3d5f340508747bfb6066b6e4e" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 32803 } } peers { permanent_uuid: "820b03e8358c483e901d2c92c2ba7f26" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 33239 } } }
I20260807 15:12:24.828815 3267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 359504a3d5f340508747bfb6066b6e4e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:12:24.829377 3350 mvcc.cc:205] Tried to move back new op lower bound from 7315929271472549888 to 7315929271439429632. Current Snapshot: MvccSnapshot[applied={T|T < 7315929271472549888}]
I20260807 15:12:24.833688 3346 catalog_manager.cc:1347] Generated new cluster ID: a70e766fd07943dcb801265d75fd12c3
I20260807 15:12:24.833801 3346 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:12:24.867729 3346 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:12:24.872165 3346 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:12:24.900157 3346 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 820b03e8358c483e901d2c92c2ba7f26: Generated new TSK 0
I20260807 15:12:24.901886 3346 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:12:25.638718 3289 catalog_manager.cc:1259] Loaded cluster ID: a70e766fd07943dcb801265d75fd12c3
I20260807 15:12:25.638844 3289 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 359504a3d5f340508747bfb6066b6e4e: loading cluster ID for follower catalog manager: success
I20260807 15:12:25.641472 3289 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 359504a3d5f340508747bfb6066b6e4e: acquiring CA information for follower catalog manager: success
I20260807 15:12:25.643551 3289 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 359504a3d5f340508747bfb6066b6e4e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:12:25.644979 3319 catalog_manager.cc:1259] Loaded cluster ID: a70e766fd07943dcb801265d75fd12c3
I20260807 15:12:25.645128 3319 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c35d1a3656144668bac24591aca22652: loading cluster ID for follower catalog manager: success
I20260807 15:12:25.648185 3319 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c35d1a3656144668bac24591aca22652: acquiring CA information for follower catalog manager: success
I20260807 15:12:25.650167 3319 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c35d1a3656144668bac24591aca22652: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:12:25.983734 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 2916
I20260807 15:12:26.000439 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 3033
I20260807 15:12:26.016402 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 3147
2026-08-07T15:12:26Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3498 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:12:26 dist-test-slave-3kk6 krb5kdc[3369](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:12:26 dist-test-slave-3kk6 krb5kdc[3369](info): set up 2 sockets
Aug 07 15:12:26 dist-test-slave-3kk6 krb5kdc[3369](info): commencing operation
krb5kdc: starting...
W20260807 15:12:27.325309 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.280s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:12:27 dist-test-slave-3kk6 krb5kdc[3369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:12:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:12:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:12:27.466118 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:40701
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:36937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:40701
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:27.588940 3383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:27.589275 3383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:27.589319 3383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:27.593634 3383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:12:27.593758 3383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:27.593797 3383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:27.593840 3383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:12:27.593871 3383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:12:27.599797 3383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:36937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:40701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:40701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:27.601053 3383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:27.602206 3383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:27.611676 3396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:27.611676 3392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:27.611759 3391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:27.611886 3383 server_base.cc:1034] running on GCE node
I20260807 15:12:27.612665 3383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:27.613961 3383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:27.615161 3383 hybrid_clock.cc:648] HybridClock initialized: now 1786115547615125 us; error 56 us; skew 500 ppm
Aug 07 15:12:27 dist-test-slave-3kk6 krb5kdc[3369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115547, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:27.618369 3383 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:12:27.619423 3383 webserver.cc:460] Webserver started at http://127.15.226.126:41217/ using document root <none> and password file <none>
I20260807 15:12:27.619995 3383 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:27.620059 3383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:27.620251 3383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:27.622463 3383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "e6f36b45e24c4461b5a9c13fb7e8be3d"
format_stamp: "Formatted at 2026-08-07 15:12:27 on dist-test-slave-3kk6"
server_key: "5b1c92f5c73c478332854b636cdd4d74"
server_key_iv: "a2c5d1508212b6918515257c88cba530"
server_key_version: "encryptionkey@0"
I20260807 15:12:27.623163 3383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "e6f36b45e24c4461b5a9c13fb7e8be3d"
format_stamp: "Formatted at 2026-08-07 15:12:27 on dist-test-slave-3kk6"
server_key: "5b1c92f5c73c478332854b636cdd4d74"
server_key_iv: "a2c5d1508212b6918515257c88cba530"
server_key_version: "encryptionkey@0"
I20260807 15:12:27.627307 3383 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:12:27.629915 3406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:27.631435 3383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:12:27.631558 3383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "e6f36b45e24c4461b5a9c13fb7e8be3d"
format_stamp: "Formatted at 2026-08-07 15:12:27 on dist-test-slave-3kk6"
server_key: "5b1c92f5c73c478332854b636cdd4d74"
server_key_iv: "a2c5d1508212b6918515257c88cba530"
server_key_version: "encryptionkey@0"
I20260807 15:12:27.631678 3383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:27.652866 3383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:27.656659 3383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:27.656874 3383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:27.665382 3383 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:40701
I20260807 15:12:27.665414 3488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:40701 every 8 connection(s)
I20260807 15:12:27.666515 3383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:12:27.669610 3489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:27.675455 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 3383
I20260807 15:12:27.675624 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:12:27.675918 16265 external_mini_cluster.cc:1428] Setting key 7136b8dfed166da918af614946f7675e
I20260807 15:12:27.676157 3489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6f36b45e24c4461b5a9c13fb7e8be3d: Bootstrap starting.
I20260807 15:12:27.679286 3489 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6f36b45e24c4461b5a9c13fb7e8be3d: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:27.680294 3489 log.cc:824] T 00000000000000000000000000000000 P e6f36b45e24c4461b5a9c13fb7e8be3d: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:27.682971 3489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6f36b45e24c4461b5a9c13fb7e8be3d: No bootstrap required, opened a new log
Aug 07 15:12:27 dist-test-slave-3kk6 krb5kdc[3369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:12:27.687336 3489 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6f36b45e24c4461b5a9c13fb7e8be3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6f36b45e24c4461b5a9c13fb7e8be3d" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 40701 } }
I20260807 15:12:27.687733 3489 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6f36b45e24c4461b5a9c13fb7e8be3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:27.687834 3489 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6f36b45e24c4461b5a9c13fb7e8be3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6f36b45e24c4461b5a9c13fb7e8be3d, State: Initialized, Role: FOLLOWER
I20260807 15:12:27.688501 3489 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6f36b45e24c4461b5a9c13fb7e8be3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6f36b45e24c4461b5a9c13fb7e8be3d" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 40701 } }
I20260807 15:12:27.688740 3489 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6f36b45e24c4461b5a9c13fb7e8be3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:12:27.688818 3489 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6f36b45e24c4461b5a9c13fb7e8be3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:12:27.688947 3489 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6f36b45e24c4461b5a9c13fb7e8be3d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:27.690377 3489 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6f36b45e24c4461b5a9c13fb7e8be3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6f36b45e24c4461b5a9c13fb7e8be3d" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 40701 } }
I20260807 15:12:27.690843 3489 leader_election.cc:302] T 00000000000000000000000000000000 P e6f36b45e24c4461b5a9c13fb7e8be3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6f36b45e24c4461b5a9c13fb7e8be3d; no voters:
I20260807 15:12:27.691332 3489 leader_election.cc:288] T 00000000000000000000000000000000 P e6f36b45e24c4461b5a9c13fb7e8be3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:12:27.691475 3494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6f36b45e24c4461b5a9c13fb7e8be3d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:12:27.691761 3494 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6f36b45e24c4461b5a9c13fb7e8be3d [term 1 LEADER]: Becoming Leader. State: Replica: e6f36b45e24c4461b5a9c13fb7e8be3d, State: Running, Role: LEADER
I20260807 15:12:27.692333 3494 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6f36b45e24c4461b5a9c13fb7e8be3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6f36b45e24c4461b5a9c13fb7e8be3d" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 40701 } }
I20260807 15:12:27.692883 3489 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6f36b45e24c4461b5a9c13fb7e8be3d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:12:27.693790 3492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:27.677484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52456 (local address 127.15.226.126:40701)
0807 15:12:27.677994 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:27.678016 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:27.678065 (+ 49us) server_negotiation.cc:407] Connection header received
0807 15:12:27.678994 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:27.679020 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:27.679497 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:27.679908 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:27.680735 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:27.682139 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:27.683292 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:27.684040 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:27.684219 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:27.687131 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:27.687139 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:27.687158 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:27.687199 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:27.689708 (+ 2509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:27.690281 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:27.690291 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:27.690301 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:27.690495 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:27.690952 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:27.690956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:27.690958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:27.691614 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:27.691849 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:27.692356 (+ 507us) server_negotiation.cc:299] Negotiation successful
0807 15:12:27.692709 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":163,"threads_started":1}
I20260807 15:12:27.693783 3496 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6f36b45e24c4461b5a9c13fb7e8be3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6f36b45e24c4461b5a9c13fb7e8be3d. Latest consensus state: current_term: 1 leader_uuid: "e6f36b45e24c4461b5a9c13fb7e8be3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6f36b45e24c4461b5a9c13fb7e8be3d" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 40701 } } }
I20260807 15:12:27.693990 3496 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6f36b45e24c4461b5a9c13fb7e8be3d [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:27.694505 3506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:12:27.694765 3495 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6f36b45e24c4461b5a9c13fb7e8be3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6f36b45e24c4461b5a9c13fb7e8be3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6f36b45e24c4461b5a9c13fb7e8be3d" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 40701 } } }
I20260807 15:12:27.694990 3495 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6f36b45e24c4461b5a9c13fb7e8be3d [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:27.698143 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:12:27.698457 3506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:12:27.705066 3506 catalog_manager.cc:1347] Generated new cluster ID: eee957752aed4684b94ca87e320a057a
I20260807 15:12:27.705176 3506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:12:27.721403 3506 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:12:27.722532 3506 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:12:27.730765 3506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6f36b45e24c4461b5a9c13fb7e8be3d: Generated new TSK 0
I20260807 15:12:27.731596 3506 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:12:27.755599 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 3383
2026-08-07T15:12:27Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1741 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:12:27 dist-test-slave-3kk6 krb5kdc[3522](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:12:27 dist-test-slave-3kk6 krb5kdc[3522](info): set up 2 sockets
Aug 07 15:12:27 dist-test-slave-3kk6 krb5kdc[3522](info): commencing operation
krb5kdc: starting...
W20260807 15:12:29.016322 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:12:29 dist-test-slave-3kk6 krb5kdc[3522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:12:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:12:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:12:29.158239 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:46183
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:46183
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:29.275128 3542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:29.275496 3542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:29.275578 3542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:29.279755 3542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:12:29.279881 3542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:29.279927 3542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:29.279970 3542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:12:29.280007 3542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:12:29.285645 3542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:46183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:46183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:29.286942 3542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:29.288053 3542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:29.297322 3551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:29.297343 3547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:29.297415 3542 server_base.cc:1034] running on GCE node
W20260807 15:12:29.297343 3548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:29.298285 3542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:29.299647 3542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:29.300843 3542 hybrid_clock.cc:648] HybridClock initialized: now 1786115549300825 us; error 40 us; skew 500 ppm
Aug 07 15:12:29 dist-test-slave-3kk6 krb5kdc[3522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115549, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:29.303830 3542 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:12:29.304829 3542 webserver.cc:460] Webserver started at http://127.15.226.126:35357/ using document root <none> and password file <none>
I20260807 15:12:29.305377 3542 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:29.305449 3542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:29.305627 3542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:29.307945 3542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "fdc5c807613c4ce5a462f9b6644a8486"
format_stamp: "Formatted at 2026-08-07 15:12:29 on dist-test-slave-3kk6"
server_key: "5d05a3742c561cdf587ecccc3179aa9b"
server_key_iv: "7d2387efd6ab98088f5ced80d4eda109"
server_key_version: "encryptionkey@0"
I20260807 15:12:29.308614 3542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "fdc5c807613c4ce5a462f9b6644a8486"
format_stamp: "Formatted at 2026-08-07 15:12:29 on dist-test-slave-3kk6"
server_key: "5d05a3742c561cdf587ecccc3179aa9b"
server_key_iv: "7d2387efd6ab98088f5ced80d4eda109"
server_key_version: "encryptionkey@0"
I20260807 15:12:29.312752 3542 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:12:29.315618 3561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:29.317029 3542 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:12:29.317183 3542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "fdc5c807613c4ce5a462f9b6644a8486"
format_stamp: "Formatted at 2026-08-07 15:12:29 on dist-test-slave-3kk6"
server_key: "5d05a3742c561cdf587ecccc3179aa9b"
server_key_iv: "7d2387efd6ab98088f5ced80d4eda109"
server_key_version: "encryptionkey@0"
I20260807 15:12:29.317313 3542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:29.351318 3542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:29.355337 3542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:29.355587 3542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:29.363670 3542 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:46183
I20260807 15:12:29.363993 3646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:46183 every 8 connection(s)
I20260807 15:12:29.364888 3542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:12:29.367143 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 3542
I20260807 15:12:29.367290 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:12:29.367640 16265 external_mini_cluster.cc:1428] Setting key 772f895e067c36f57254e6e61b5380b1
I20260807 15:12:29.368259 3647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:29.374958 3647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486: Bootstrap starting.
Aug 07 15:12:29 dist-test-slave-3kk6 krb5kdc[3522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:12:29.377564 3647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:29.378504 3647 log.cc:824] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:29.380642 3647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486: No bootstrap required, opened a new log
I20260807 15:12:29.382758 3651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:29.369286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44286 (local address 127.15.226.126:46183)
0807 15:12:29.369805 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:29.369821 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:29.369869 (+ 48us) server_negotiation.cc:407] Connection header received
0807 15:12:29.370572 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:29.370588 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:29.371030 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:29.371406 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:29.372356 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:29.373400 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:29.374299 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:29.374874 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:29.375050 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:29.377770 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:29.377776 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:29.377790 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:29.377820 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:29.379662 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:29.380151 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:29.380156 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:29.380161 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:29.380279 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:29.380589 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:29.380593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:29.380595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:29.381074 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:29.381294 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:29.381611 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 15:12:29.381940 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":166,"threads_started":1}
I20260807 15:12:29.384563 3647 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc5c807613c4ce5a462f9b6644a8486" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46183 } }
I20260807 15:12:29.384847 3647 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:29.384989 3647 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdc5c807613c4ce5a462f9b6644a8486, State: Initialized, Role: FOLLOWER
I20260807 15:12:29.385461 3647 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc5c807613c4ce5a462f9b6644a8486" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46183 } }
I20260807 15:12:29.385635 3647 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:12:29.385722 3647 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:12:29.385854 3647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:29.387281 3647 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc5c807613c4ce5a462f9b6644a8486" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46183 } }
I20260807 15:12:29.387683 3647 leader_election.cc:302] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdc5c807613c4ce5a462f9b6644a8486; no voters:
I20260807 15:12:29.388109 3647 leader_election.cc:288] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:12:29.388231 3653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:12:29.388567 3653 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 1 LEADER]: Becoming Leader. State: Replica: fdc5c807613c4ce5a462f9b6644a8486, State: Running, Role: LEADER
I20260807 15:12:29.388952 3653 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc5c807613c4ce5a462f9b6644a8486" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46183 } }
I20260807 15:12:29.389622 3647 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:12:29.390960 3654 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdc5c807613c4ce5a462f9b6644a8486" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc5c807613c4ce5a462f9b6644a8486" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46183 } } }
I20260807 15:12:29.391105 3654 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:29.391465 3655 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdc5c807613c4ce5a462f9b6644a8486. Latest consensus state: current_term: 1 leader_uuid: "fdc5c807613c4ce5a462f9b6644a8486" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc5c807613c4ce5a462f9b6644a8486" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46183 } } }
I20260807 15:12:29.391553 3667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:12:29.391571 3655 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:29.394917 3667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:12:29.401187 3667 catalog_manager.cc:1347] Generated new cluster ID: 970ee3cf882940cca581d7917f8e935c
I20260807 15:12:29.401309 3667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:12:29.418260 3667 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:12:29.419652 3667 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:12:29.425603 3667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486: Generated new TSK 0
I20260807 15:12:29.426456 3667 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.226.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
I20260807 15:12:29.462404 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:46183
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:29.588001 3685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:29.588505 3685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:29.588591 3685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:29.593462 3685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:29.593587 3685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:29.593691 3685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:12:29.599428 3685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:46183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:29.600761 3685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:29.601972 3685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:29.612679 3691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:29.612783 3690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:29.612749 3685 server_base.cc:1034] running on GCE node
W20260807 15:12:29.612679 3693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:29.613513 3685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:29.614285 3685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:29.615487 3685 hybrid_clock.cc:648] HybridClock initialized: now 1786115549615470 us; error 40 us; skew 500 ppm
Aug 07 15:12:29 dist-test-slave-3kk6 krb5kdc[3522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115549, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:29.618498 3685 init.cc:377] Logged in from keytab as kudu/127.15.226.65@KRBTEST.COM (short username kudu)
I20260807 15:12:29.619513 3685 webserver.cc:460] Webserver started at http://127.15.226.65:37363/ using document root <none> and password file <none>
I20260807 15:12:29.620093 3685 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:29.620160 3685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:29.620363 3685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:29.622884 3685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "53f7598c906449ddad9b0f69ca2278ef"
format_stamp: "Formatted at 2026-08-07 15:12:29 on dist-test-slave-3kk6"
server_key: "f4a6173e00df66104e98268b61984ba7"
server_key_iv: "9d34c4a154e146af42cdc4f5b4023ad7"
server_key_version: "encryptionkey@0"
I20260807 15:12:29.623479 3685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "53f7598c906449ddad9b0f69ca2278ef"
format_stamp: "Formatted at 2026-08-07 15:12:29 on dist-test-slave-3kk6"
server_key: "f4a6173e00df66104e98268b61984ba7"
server_key_iv: "9d34c4a154e146af42cdc4f5b4023ad7"
server_key_version: "encryptionkey@0"
I20260807 15:12:29.627525 3685 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 15:12:29.630095 3702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:29.631585 3685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:12:29.631750 3685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "53f7598c906449ddad9b0f69ca2278ef"
format_stamp: "Formatted at 2026-08-07 15:12:29 on dist-test-slave-3kk6"
server_key: "f4a6173e00df66104e98268b61984ba7"
server_key_iv: "9d34c4a154e146af42cdc4f5b4023ad7"
server_key_version: "encryptionkey@0"
I20260807 15:12:29.631897 3685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:29.659909 3685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:29.663599 3685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:29.663838 3685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:29.664525 3685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:29.665500 3685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:29.665551 3685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:29.665628 3685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:29.665663 3685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:29.675860 3685 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:40159
I20260807 15:12:29.675881 3883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:40159 every 8 connection(s)
I20260807 15:12:29.676936 3685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:12:29.681661 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 3685
I20260807 15:12:29.681806 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:12:29.682211 16265 external_mini_cluster.cc:1428] Setting key de8c3d142af54c3a64b20ca14bb2618d
Aug 07 15:12:29 dist-test-slave-3kk6 krb5kdc[3522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115549, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.226.66@KRBTEST.COM; defaulting to no policy
I20260807 15:12:29.693565 3651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:29.679268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:54681 (local address 127.15.226.126:46183)
0807 15:12:29.679448 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:29.679452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:29.680424 (+ 972us) server_negotiation.cc:407] Connection header received
0807 15:12:29.682063 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:29.682068 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:29.682140 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:29.682254 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:29.683830 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:29.684644 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:29.685992 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:29.686394 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:29.686529 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:29.689406 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:29.689412 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:29.689416 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:29.689452 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:29.691172 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:29.691873 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:29.691877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:29.691878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:29.691964 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:29.692348 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:29.692353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:29.692354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:29.692561 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:29.692679 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:29.693227 (+ 548us) server_negotiation.cc:299] Negotiation successful
0807 15:12:29.693388 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 15:12:29.694585 3886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:29.679686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.65:54681)
0807 15:12:29.680235 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:29.680286 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:29.681661 (+ 1375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:29.682489 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:29.682497 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:29.683076 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:29.683645 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:29.683662 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:29.684813 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:29.684816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:29.685844 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:29.685856 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:29.686588 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:29.686592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:29.686839 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:29.687557 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:29.687575 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:29.689205 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:29.691318 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:29.691325 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:29.691344 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:29.691734 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:29.692076 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:29.692079 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:29.692081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:29.692242 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:29.692679 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:29.692687 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:29.692974 (+ 287us) client_negotiation.cc:769] Sending connection context
0807 15:12:29.693184 (+ 210us) client_negotiation.cc:240] Negotiation successful
0807 15:12:29.693576 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":152,"threads_started":1}
I20260807 15:12:29.696122 3884 heartbeater.cc:347] Connected to a master server at 127.15.226.126:46183
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.66@KRBTEST.COM" created.
I20260807 15:12:29.696468 3884 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:29.697140 3884 heartbeater.cc:511] Master 127.15.226.126:46183 requested a full tablet report, sending...
I20260807 15:12:29.699108 3589 ts_manager.cc:194] Registered new tserver with Master: 53f7598c906449ddad9b0f69ca2278ef (127.15.226.65:40159)
I20260807 15:12:29.700832 3589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.65@KRBTEST.COM'} at 127.15.226.65:54681
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
I20260807 15:12:29.714147 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:46183
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:29.835394 3890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:29.835723 3890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:29.835805 3890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:29.840212 3890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:29.840306 3890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:29.840399 3890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:12:29.846093 3890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:46183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:29.847448 3890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:29.848572 3890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:12:29.859107 3890 server_base.cc:1034] running on GCE node
W20260807 15:12:29.859052 3898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:29.859052 3897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:29.859081 3900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:29.859865 3890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:29.860563 3890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:29.861750 3890 hybrid_clock.cc:648] HybridClock initialized: now 1786115549861731 us; error 36 us; skew 500 ppm
Aug 07 15:12:29 dist-test-slave-3kk6 krb5kdc[3522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115549, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:29.865001 3890 init.cc:377] Logged in from keytab as kudu/127.15.226.66@KRBTEST.COM (short username kudu)
I20260807 15:12:29.865988 3890 webserver.cc:460] Webserver started at http://127.15.226.66:34541/ using document root <none> and password file <none>
I20260807 15:12:29.866595 3890 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:29.866727 3890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:29.867020 3890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:29.869292 3890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "5cd7e7e149cf45f5a644b92416699061"
format_stamp: "Formatted at 2026-08-07 15:12:29 on dist-test-slave-3kk6"
server_key: "c768243eab5c25f22a4c992a18fa5735"
server_key_iv: "d81becac286a08a3a5dcf37358603962"
server_key_version: "encryptionkey@0"
I20260807 15:12:29.869946 3890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "5cd7e7e149cf45f5a644b92416699061"
format_stamp: "Formatted at 2026-08-07 15:12:29 on dist-test-slave-3kk6"
server_key: "c768243eab5c25f22a4c992a18fa5735"
server_key_iv: "d81becac286a08a3a5dcf37358603962"
server_key_version: "encryptionkey@0"
I20260807 15:12:29.874123 3890 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260807 15:12:29.876780 3912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:29.878127 3890 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:12:29.878284 3890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "5cd7e7e149cf45f5a644b92416699061"
format_stamp: "Formatted at 2026-08-07 15:12:29 on dist-test-slave-3kk6"
server_key: "c768243eab5c25f22a4c992a18fa5735"
server_key_iv: "d81becac286a08a3a5dcf37358603962"
server_key_version: "encryptionkey@0"
I20260807 15:12:29.878417 3890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:29.902402 3890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:29.906057 3890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:29.906306 3890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:29.907065 3890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:29.908028 3890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:29.908107 3890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:29.908186 3890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:29.908237 3890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:29.918375 3890 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:39565
I20260807 15:12:29.918395 4088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:39565 every 8 connection(s)
I20260807 15:12:29.919478 3890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:12:29.923013 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 3890
I20260807 15:12:29.923162 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:12:29.923460 16265 external_mini_cluster.cc:1428] Setting key ed420e1481760fd80066b30032d07d1f
Aug 07 15:12:29 dist-test-slave-3kk6 krb5kdc[3522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115549, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.226.67@KRBTEST.COM; defaulting to no policy
I20260807 15:12:29.935855 3651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:29.921431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:38449 (local address 127.15.226.126:46183)
0807 15:12:29.921598 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:29.921602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:29.922599 (+ 997us) server_negotiation.cc:407] Connection header received
0807 15:12:29.924387 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:29.924390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:29.924436 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:29.924516 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:29.925963 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:29.926825 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:29.928086 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:29.928324 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:29.928470 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:29.931615 (+ 3145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:29.931619 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:29.931621 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:29.931650 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:29.933308 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:29.933942 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:29.933946 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:29.933948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:29.934045 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:29.934437 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:29.934440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:29.934442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:29.934705 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:29.934832 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:29.935484 (+ 652us) server_negotiation.cc:299] Negotiation successful
0807 15:12:29.935668 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 15:12:29.936867 4094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:29.921865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.66:38449)
0807 15:12:29.922414 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:29.922469 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:29.924149 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:29.924706 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:29.924715 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:29.925282 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:29.925812 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:29.925827 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:29.926974 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:29.926976 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:29.927937 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:29.927947 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:29.928507 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:29.928511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:29.928711 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:29.929404 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:29.929422 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:29.931382 (+ 1960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:29.933468 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:29.933478 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:29.933498 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:29.933813 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:29.934149 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:29.934152 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:29.934154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:29.934323 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:29.934832 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:29.934841 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:29.935209 (+ 368us) client_negotiation.cc:769] Sending connection context
0807 15:12:29.935430 (+ 221us) client_negotiation.cc:240] Negotiation successful
0807 15:12:29.935830 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":177,"threads_started":1}
I20260807 15:12:29.938367 4090 heartbeater.cc:347] Connected to a master server at 127.15.226.126:46183
I20260807 15:12:29.938716 4090 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.67@KRBTEST.COM" created.
I20260807 15:12:29.939481 4090 heartbeater.cc:511] Master 127.15.226.126:46183 requested a full tablet report, sending...
I20260807 15:12:29.940809 3589 ts_manager.cc:194] Registered new tserver with Master: 5cd7e7e149cf45f5a644b92416699061 (127.15.226.66:39565)
I20260807 15:12:29.941534 3589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.66@KRBTEST.COM'} at 127.15.226.66:38449
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
I20260807 15:12:29.958501 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:46183
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:30.083140 4098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:30.083479 4098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:30.083563 4098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:30.087756 4098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:30.087874 4098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:30.087977 4098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:12:30.093971 4098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:46183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:30.095419 4098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:30.096988 4098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:30.109512 4105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:30.109494 4111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:30.109561 4113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:30.109694 4098 server_base.cc:1034] running on GCE node
I20260807 15:12:30.110378 4098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:30.111207 4098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:30.112406 4098 hybrid_clock.cc:648] HybridClock initialized: now 1786115550112396 us; error 34 us; skew 500 ppm
Aug 07 15:12:30 dist-test-slave-3kk6 krb5kdc[3522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115550, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:30.115532 4098 init.cc:377] Logged in from keytab as kudu/127.15.226.67@KRBTEST.COM (short username kudu)
I20260807 15:12:30.116499 4098 webserver.cc:460] Webserver started at http://127.15.226.67:40823/ using document root <none> and password file <none>
I20260807 15:12:30.117053 4098 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:30.117115 4098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:30.117290 4098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:30.119514 4098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "e5491767f24d4ac5812d23cf91db8d0a"
format_stamp: "Formatted at 2026-08-07 15:12:30 on dist-test-slave-3kk6"
server_key: "2c6a399c214f7bb4cb9bd04690b07931"
server_key_iv: "f10174211d2c301d86b53729415a44bc"
server_key_version: "encryptionkey@0"
I20260807 15:12:30.120056 4098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "e5491767f24d4ac5812d23cf91db8d0a"
format_stamp: "Formatted at 2026-08-07 15:12:30 on dist-test-slave-3kk6"
server_key: "2c6a399c214f7bb4cb9bd04690b07931"
server_key_iv: "f10174211d2c301d86b53729415a44bc"
server_key_version: "encryptionkey@0"
I20260807 15:12:30.124092 4098 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:12:30.126497 4126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:30.127811 4098 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:12:30.127952 4098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "e5491767f24d4ac5812d23cf91db8d0a"
format_stamp: "Formatted at 2026-08-07 15:12:30 on dist-test-slave-3kk6"
server_key: "2c6a399c214f7bb4cb9bd04690b07931"
server_key_iv: "f10174211d2c301d86b53729415a44bc"
server_key_version: "encryptionkey@0"
I20260807 15:12:30.128068 4098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:30.148109 4098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:30.151830 4098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:30.152065 4098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:30.152748 4098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:30.153662 4098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:30.153713 4098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:30.153756 4098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:30.153813 4098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:30.163792 4098 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:35353
I20260807 15:12:30.163813 4306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:35353 every 8 connection(s)
I20260807 15:12:30.164808 4098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:12:30.167865 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 4098
I20260807 15:12:30.168017 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:12:30.168399 16265 external_mini_cluster.cc:1428] Setting key 064013b60b65519ee1b1fa6cba9a531b
Aug 07 15:12:30 dist-test-slave-3kk6 krb5kdc[3522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115550, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:12:30.180150 3651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:30.166854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:58809 (local address 127.15.226.126:46183)
0807 15:12:30.167023 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:30.167028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:30.168122 (+ 1094us) server_negotiation.cc:407] Connection header received
0807 15:12:30.170203 (+ 2081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:30.170206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:30.170259 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:30.170349 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:30.171787 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.172325 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.173486 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.173701 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.173783 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:30.176464 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:30.176468 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:30.176470 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:30.176498 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:30.178094 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:30.178639 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:30.178642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:30.178643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:30.178775 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:30.179094 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:30.179096 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:30.179098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:30.179281 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:30.179356 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:30.179882 (+ 526us) server_negotiation.cc:299] Negotiation successful
0807 15:12:30.180013 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 15:12:30.181039 4310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:30.167263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.67:58809)
0807 15:12:30.167913 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:30.167985 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:30.169859 (+ 1874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:30.170567 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:30.170575 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:30.171105 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:30.171630 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:30.171644 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.172441 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.172444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:30.173370 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:30.173380 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.173841 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.173843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:30.173988 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:30.174622 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:30.174639 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:30.176262 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:30.178226 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:30.178235 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:30.178254 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:30.178538 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:30.178857 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:30.178859 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:30.178861 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:30.179005 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:30.179370 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:30.179376 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:30.179645 (+ 269us) client_negotiation.cc:769] Sending connection context
0807 15:12:30.179816 (+ 171us) client_negotiation.cc:240] Negotiation successful
0807 15:12:30.180100 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":116,"threads_started":1}
I20260807 15:12:30.182461 4308 heartbeater.cc:347] Connected to a master server at 127.15.226.126:46183
I20260807 15:12:30.182773 4308 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:30.183512 4308 heartbeater.cc:511] Master 127.15.226.126:46183 requested a full tablet report, sending...
I20260807 15:12:30.184620 3589 ts_manager.cc:194] Registered new tserver with Master: e5491767f24d4ac5812d23cf91db8d0a (127.15.226.67:35353)
I20260807 15:12:30.185242 3589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.67@KRBTEST.COM'} at 127.15.226.67:58809
I20260807 15:12:30.195677 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:12:30.206368 3651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:30.197968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44296 (local address 127.15.226.126:46183)
0807 15:12:30.198137 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:30.198141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:30.198298 (+ 157us) server_negotiation.cc:407] Connection header received
0807 15:12:30.198406 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:30.198409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:30.198452 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:30.198529 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:30.199370 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.199978 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.200869 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.201110 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.201208 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:30.202768 (+ 1560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:30.202772 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:30.202774 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:30.202801 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:30.204445 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:30.204956 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:30.204960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:30.204961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:30.205042 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:30.205342 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:30.205346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:30.205348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:30.205595 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:30.205745 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:30.206042 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 15:12:30.206174 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 15:12:30.209872 3589 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44296:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:12:30.213249 3589 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:12:30.229188 4328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:30.223284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56658 (local address 127.15.226.66:39565)
0807 15:12:30.223630 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:30.223644 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:30.223670 (+ 26us) server_negotiation.cc:407] Connection header received
0807 15:12:30.223767 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:30.223779 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:30.224014 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:30.224180 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:12:30.224713 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.225861 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.227570 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.228364 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.228482 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:30.228654 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:30.228912 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 15:12:30.229018 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":145,"threads_started":1}
I20260807 15:12:30.229184 4322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:30.222728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:39565 (local address 127.0.0.1:56658)
0807 15:12:30.223245 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:30.223299 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:30.223447 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:30.224425 (+ 978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:30.224432 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:30.224457 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:12:30.224583 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:30.224588 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.226008 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.226011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:30.227424 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:30.227436 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.228483 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.228488 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:30.228680 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:30.228731 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 15:12:30.228843 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 15:12:30.228990 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":37,"thread_start_us":108,"threads_started":1}
I20260807 15:12:30.229188 4329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:30.222806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:40159 (local address 127.0.0.1:38180)
0807 15:12:30.223846 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:30.223860 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:30.223967 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:30.224799 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:30.224802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:30.224820 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:12:30.224930 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:30.224939 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.226281 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.226288 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:30.227263 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:30.227272 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.228473 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.228477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:30.228671 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:30.228731 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 15:12:30.228844 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 15:12:30.228996 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":980,"mutex_wait_us":65,"thread_start_us":114,"threads_started":1}
I20260807 15:12:30.229947 4324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:30.223537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:35353 (local address 127.0.0.1:37566)
0807 15:12:30.225080 (+ 1543us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:30.225092 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:30.225198 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:30.225616 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:30.225619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:30.225637 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:12:30.225753 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:30.225763 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.227251 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.227255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:30.228281 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:30.228291 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.229564 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.229567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:30.229642 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:30.229659 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 15:12:30.229759 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 15:12:30.229828 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1466,"thread_start_us":101,"threads_started":1}
I20260807 15:12:30.230064 4332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:30.223845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37566 (local address 127.15.226.67:35353)
0807 15:12:30.224251 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:30.224263 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:30.225191 (+ 928us) server_negotiation.cc:407] Connection header received
0807 15:12:30.225327 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:30.225331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:30.225473 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:30.225626 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:12:30.225879 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.227100 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.228407 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.229281 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.229405 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:30.229510 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:30.229795 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 15:12:30.229892 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":194,"threads_started":1}
I20260807 15:12:30.230141 4327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:30.222869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38180 (local address 127.15.226.65:40159)
0807 15:12:30.224370 (+ 1501us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:30.224374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:30.224395 (+ 21us) server_negotiation.cc:407] Connection header received
0807 15:12:30.224497 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:30.224504 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:30.224662 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:30.224821 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:12:30.225090 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.226162 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.227406 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.228296 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.229230 (+ 934us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:30.229836 (+ 606us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:30.229888 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 15:12:30.229995 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1408,"thread_start_us":94,"threads_started":1}
I20260807 15:12:30.233088 3982 tablet_service.cc:1467] Processing CreateTablet for tablet d44ff192dcc44035970773e483cc7e24 (DEFAULT_TABLE table=test-table [id=5fcd8343f3a945fbb168f0c6436c7e36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:12:30.233088 4201 tablet_service.cc:1467] Processing CreateTablet for tablet d44ff192dcc44035970773e483cc7e24 (DEFAULT_TABLE table=test-table [id=5fcd8343f3a945fbb168f0c6436c7e36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:12:30.233088 3779 tablet_service.cc:1467] Processing CreateTablet for tablet d44ff192dcc44035970773e483cc7e24 (DEFAULT_TABLE table=test-table [id=5fcd8343f3a945fbb168f0c6436c7e36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:12:30.234293 3779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d44ff192dcc44035970773e483cc7e24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:30.234308 4201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d44ff192dcc44035970773e483cc7e24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:30.234305 3982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d44ff192dcc44035970773e483cc7e24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:30.242129 4333 tablet_bootstrap.cc:492] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef: Bootstrap starting.
I20260807 15:12:30.242219 4334 tablet_bootstrap.cc:492] T d44ff192dcc44035970773e483cc7e24 P e5491767f24d4ac5812d23cf91db8d0a: Bootstrap starting.
I20260807 15:12:30.243003 4335 tablet_bootstrap.cc:492] T d44ff192dcc44035970773e483cc7e24 P 5cd7e7e149cf45f5a644b92416699061: Bootstrap starting.
I20260807 15:12:30.244258 4333 tablet_bootstrap.cc:654] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:30.244935 4334 tablet_bootstrap.cc:654] T d44ff192dcc44035970773e483cc7e24 P e5491767f24d4ac5812d23cf91db8d0a: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:30.245123 4333 log.cc:824] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:30.245594 4335 tablet_bootstrap.cc:654] T d44ff192dcc44035970773e483cc7e24 P 5cd7e7e149cf45f5a644b92416699061: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:30.245887 4334 log.cc:824] T d44ff192dcc44035970773e483cc7e24 P e5491767f24d4ac5812d23cf91db8d0a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:30.246444 4335 log.cc:824] T d44ff192dcc44035970773e483cc7e24 P 5cd7e7e149cf45f5a644b92416699061: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:30.248103 4334 tablet_bootstrap.cc:492] T d44ff192dcc44035970773e483cc7e24 P e5491767f24d4ac5812d23cf91db8d0a: No bootstrap required, opened a new log
I20260807 15:12:30.248102 4333 tablet_bootstrap.cc:492] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef: No bootstrap required, opened a new log
I20260807 15:12:30.248313 4335 tablet_bootstrap.cc:492] T d44ff192dcc44035970773e483cc7e24 P 5cd7e7e149cf45f5a644b92416699061: No bootstrap required, opened a new log
I20260807 15:12:30.248318 4334 ts_tablet_manager.cc:1397] T d44ff192dcc44035970773e483cc7e24 P e5491767f24d4ac5812d23cf91db8d0a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 15:12:30.248318 4333 ts_tablet_manager.cc:1397] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 15:12:30.248517 4335 ts_tablet_manager.cc:1397] T d44ff192dcc44035970773e483cc7e24 P 5cd7e7e149cf45f5a644b92416699061: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 15:12:30.251502 4335 raft_consensus.cc:348] T d44ff192dcc44035970773e483cc7e24 P 5cd7e7e149cf45f5a644b92416699061 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } }
I20260807 15:12:30.251824 4335 raft_consensus.cc:374] T d44ff192dcc44035970773e483cc7e24 P 5cd7e7e149cf45f5a644b92416699061 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:30.251909 4335 raft_consensus.cc:729] T d44ff192dcc44035970773e483cc7e24 P 5cd7e7e149cf45f5a644b92416699061 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd7e7e149cf45f5a644b92416699061, State: Initialized, Role: FOLLOWER
I20260807 15:12:30.252441 4335 consensus_queue.cc:260] T d44ff192dcc44035970773e483cc7e24 P 5cd7e7e149cf45f5a644b92416699061 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } }
I20260807 15:12:30.252627 4334 raft_consensus.cc:348] T d44ff192dcc44035970773e483cc7e24 P e5491767f24d4ac5812d23cf91db8d0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } }
I20260807 15:12:30.252939 4334 raft_consensus.cc:374] T d44ff192dcc44035970773e483cc7e24 P e5491767f24d4ac5812d23cf91db8d0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:30.253034 4334 raft_consensus.cc:729] T d44ff192dcc44035970773e483cc7e24 P e5491767f24d4ac5812d23cf91db8d0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5491767f24d4ac5812d23cf91db8d0a, State: Initialized, Role: FOLLOWER
I20260807 15:12:30.252945 4333 raft_consensus.cc:348] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } }
I20260807 15:12:30.253235 4090 heartbeater.cc:503] Master 127.15.226.126:46183 was elected leader, sending a full tablet report...
I20260807 15:12:30.253569 4335 ts_tablet_manager.cc:1428] T d44ff192dcc44035970773e483cc7e24 P 5cd7e7e149cf45f5a644b92416699061: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 15:12:30.253492 4334 consensus_queue.cc:260] T d44ff192dcc44035970773e483cc7e24 P e5491767f24d4ac5812d23cf91db8d0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } }
I20260807 15:12:30.253216 4333 raft_consensus.cc:374] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:30.254504 4334 ts_tablet_manager.cc:1428] T d44ff192dcc44035970773e483cc7e24 P e5491767f24d4ac5812d23cf91db8d0a: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 15:12:30.254511 4333 raft_consensus.cc:729] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53f7598c906449ddad9b0f69ca2278ef, State: Initialized, Role: FOLLOWER
I20260807 15:12:30.254791 4308 heartbeater.cc:503] Master 127.15.226.126:46183 was elected leader, sending a full tablet report...
I20260807 15:12:30.255028 4333 consensus_queue.cc:260] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } }
I20260807 15:12:30.256027 4333 ts_tablet_manager.cc:1428] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260807 15:12:30.256369 3884 heartbeater.cc:503] Master 127.15.226.126:46183 was elected leader, sending a full tablet report...
W20260807 15:12:30.415892 4309 tablet.cc:2369] T d44ff192dcc44035970773e483cc7e24 P e5491767f24d4ac5812d23cf91db8d0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:12:30.420786 4092 tablet.cc:2369] T d44ff192dcc44035970773e483cc7e24 P 5cd7e7e149cf45f5a644b92416699061: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:12:30.428573 3885 tablet.cc:2369] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:12:30.508776 4346 raft_consensus.cc:482] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:12:30.509104 4346 raft_consensus.cc:504] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } }
I20260807 15:12:30.510372 4346 leader_election.cc:288] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cd7e7e149cf45f5a644b92416699061 (127.15.226.66:39565), e5491767f24d4ac5812d23cf91db8d0a (127.15.226.67:35353)
I20260807 15:12:30.516572 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:30.510819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:39565 (local address 127.15.226.65:55259)
0807 15:12:30.511148 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:30.511167 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:30.511304 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:30.511774 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:30.511777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:30.511807 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:12:30.511982 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:30.511987 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.513135 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.513139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:30.514921 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:30.514937 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.515915 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.515919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:30.516121 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:30.516164 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 15:12:30.516297 (+ 133us) client_negotiation.cc:240] Negotiation successful
0807 15:12:30.516391 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":225,"thread_start_us":138,"threads_started":1}
I20260807 15:12:30.516551 4332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:30.511452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:39193 (local address 127.15.226.67:35353)
0807 15:12:30.511610 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:30.511614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:30.511765 (+ 151us) server_negotiation.cc:407] Connection header received
0807 15:12:30.511972 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:30.511975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:30.512023 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:30.512125 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:12:30.512282 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.512996 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.515092 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.515736 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.515879 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:30.515938 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:30.516334 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 15:12:30.516420 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 15:12:30.516556 4328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:30.510909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:55259 (local address 127.15.226.66:39565)
0807 15:12:30.511058 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:30.511062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:30.511267 (+ 205us) server_negotiation.cc:407] Connection header received
0807 15:12:30.511486 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:30.511490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:30.511561 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:30.511620 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:12:30.512108 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.512931 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.515092 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.515736 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.515880 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:30.515938 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:30.516326 (+ 388us) server_negotiation.cc:299] Negotiation successful
0807 15:12:30.516428 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 15:12:30.516978 4349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:30.511271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:35353 (local address 127.15.226.65:39193)
0807 15:12:30.511632 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:30.511648 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:30.511829 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:30.512128 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:30.512130 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:30.512142 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:12:30.512213 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:30.512217 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.513135 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.513139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:30.514922 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:30.514936 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.515914 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.515919 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:30.516114 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:30.516169 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 15:12:30.516751 (+ 582us) client_negotiation.cc:240] Negotiation successful
0807 15:12:30.516848 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":275,"thread_start_us":130,"threads_started":1}
I20260807 15:12:30.517731 4232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d44ff192dcc44035970773e483cc7e24" candidate_uuid: "53f7598c906449ddad9b0f69ca2278ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5491767f24d4ac5812d23cf91db8d0a" is_pre_election: true
I20260807 15:12:30.517757 4014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d44ff192dcc44035970773e483cc7e24" candidate_uuid: "53f7598c906449ddad9b0f69ca2278ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd7e7e149cf45f5a644b92416699061" is_pre_election: true
I20260807 15:12:30.518082 4232 raft_consensus.cc:2454] T d44ff192dcc44035970773e483cc7e24 P e5491767f24d4ac5812d23cf91db8d0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53f7598c906449ddad9b0f69ca2278ef in term 0.
I20260807 15:12:30.518088 4014 raft_consensus.cc:2454] T d44ff192dcc44035970773e483cc7e24 P 5cd7e7e149cf45f5a644b92416699061 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53f7598c906449ddad9b0f69ca2278ef in term 0.
I20260807 15:12:30.518581 3704 leader_election.cc:302] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53f7598c906449ddad9b0f69ca2278ef, e5491767f24d4ac5812d23cf91db8d0a; no voters:
I20260807 15:12:30.518959 4346 raft_consensus.cc:2781] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:12:30.519060 4346 raft_consensus.cc:482] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:12:30.519105 4346 raft_consensus.cc:3037] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:30.520277 4346 raft_consensus.cc:504] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } }
I20260807 15:12:30.520689 4346 leader_election.cc:288] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef [CANDIDATE]: Term 1 election: Requested vote from peers 5cd7e7e149cf45f5a644b92416699061 (127.15.226.66:39565), e5491767f24d4ac5812d23cf91db8d0a (127.15.226.67:35353)
I20260807 15:12:30.521138 4014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d44ff192dcc44035970773e483cc7e24" candidate_uuid: "53f7598c906449ddad9b0f69ca2278ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cd7e7e149cf45f5a644b92416699061"
I20260807 15:12:30.521284 4232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d44ff192dcc44035970773e483cc7e24" candidate_uuid: "53f7598c906449ddad9b0f69ca2278ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5491767f24d4ac5812d23cf91db8d0a"
I20260807 15:12:30.521306 4014 raft_consensus.cc:3037] T d44ff192dcc44035970773e483cc7e24 P 5cd7e7e149cf45f5a644b92416699061 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:30.521453 4232 raft_consensus.cc:3037] T d44ff192dcc44035970773e483cc7e24 P e5491767f24d4ac5812d23cf91db8d0a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:30.522801 4232 raft_consensus.cc:2454] T d44ff192dcc44035970773e483cc7e24 P e5491767f24d4ac5812d23cf91db8d0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53f7598c906449ddad9b0f69ca2278ef in term 1.
I20260807 15:12:30.522881 4014 raft_consensus.cc:2454] T d44ff192dcc44035970773e483cc7e24 P 5cd7e7e149cf45f5a644b92416699061 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53f7598c906449ddad9b0f69ca2278ef in term 1.
I20260807 15:12:30.523126 3704 leader_election.cc:302] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53f7598c906449ddad9b0f69ca2278ef, e5491767f24d4ac5812d23cf91db8d0a; no voters:
I20260807 15:12:30.523417 4346 raft_consensus.cc:2781] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:12:30.523664 4346 raft_consensus.cc:686] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef [term 1 LEADER]: Becoming Leader. State: Replica: 53f7598c906449ddad9b0f69ca2278ef, State: Running, Role: LEADER
I20260807 15:12:30.524030 4346 consensus_queue.cc:237] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } }
I20260807 15:12:30.527601 3589 catalog_manager.cc:5495] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef reported cstate change: term changed from 0 to 1, leader changed from <none> to 53f7598c906449ddad9b0f69ca2278ef (127.15.226.65). New cstate: current_term: 1 leader_uuid: "53f7598c906449ddad9b0f69ca2278ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } health_report { overall_health: UNKNOWN } } }
I20260807 15:12:30.597842 4327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:30.593726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38192 (local address 127.15.226.65:40159)
0807 15:12:30.593894 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:30.593899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:30.593916 (+ 17us) server_negotiation.cc:407] Connection header received
0807 15:12:30.594070 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:30.594072 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:30.594135 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:30.594262 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:12:30.594559 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.595171 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.596186 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:30.596423 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:30.596546 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:30.596930 (+ 384us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:12:30.597407 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:12:30.597550 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:30.597616 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 15:12:30.597683 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 15:12:30.604581 4014 raft_consensus.cc:1261] T d44ff192dcc44035970773e483cc7e24 P 5cd7e7e149cf45f5a644b92416699061 [term 1 FOLLOWER]: Refusing update from remote peer 53f7598c906449ddad9b0f69ca2278ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:12:30.604610 4232 raft_consensus.cc:1261] T d44ff192dcc44035970773e483cc7e24 P e5491767f24d4ac5812d23cf91db8d0a [term 1 FOLLOWER]: Refusing update from remote peer 53f7598c906449ddad9b0f69ca2278ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:12:30.605448 4346 consensus_queue.cc:1035] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:12:30.605779 4351 consensus_queue.cc:1035] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:12:30.612646 4354 mvcc.cc:205] Tried to move back new op lower bound from 7315929295267123200 to 7315929294949036032. Current Snapshot: MvccSnapshot[applied={T|T < 7315929295267123200}]
I20260807 15:12:30.615826 4356 mvcc.cc:205] Tried to move back new op lower bound from 7315929295267123200 to 7315929294949036032. Current Snapshot: MvccSnapshot[applied={T|T < 7315929295267123200}]
I20260807 15:12:30.623677 3587 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:12:30.623869 3587 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:12:30.626564 3587 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486: Sending DeleteTablet for 3 replicas of tablet d44ff192dcc44035970773e483cc7e24
I20260807 15:12:30.627663 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 3685
I20260807 15:12:30.627727 3779 tablet_service.cc:1514] Processing DeleteTablet for tablet d44ff192dcc44035970773e483cc7e24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:12:30 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:38180
I20260807 15:12:30.627748 3982 tablet_service.cc:1514] Processing DeleteTablet for tablet d44ff192dcc44035970773e483cc7e24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:12:30 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:56658
I20260807 15:12:30.635553 4367 tablet_replica.cc:330] T d44ff192dcc44035970773e483cc7e24 P 5cd7e7e149cf45f5a644b92416699061: stopping tablet replica
I20260807 15:12:30.636045 4367 raft_consensus.cc:2229] T d44ff192dcc44035970773e483cc7e24 P 5cd7e7e149cf45f5a644b92416699061 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:12:30.636114 4201 tablet_service.cc:1514] Processing DeleteTablet for tablet d44ff192dcc44035970773e483cc7e24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:12:30 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:37566
I20260807 15:12:30.636467 4367 raft_consensus.cc:2258] T d44ff192dcc44035970773e483cc7e24 P 5cd7e7e149cf45f5a644b92416699061 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:12:30.636679 4368 tablet_replica.cc:330] T d44ff192dcc44035970773e483cc7e24 P e5491767f24d4ac5812d23cf91db8d0a: stopping tablet replica
I20260807 15:12:30.637116 4368 raft_consensus.cc:2229] T d44ff192dcc44035970773e483cc7e24 P e5491767f24d4ac5812d23cf91db8d0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:12:30.637535 4368 raft_consensus.cc:2258] T d44ff192dcc44035970773e483cc7e24 P e5491767f24d4ac5812d23cf91db8d0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:12:30.639168 4367 ts_tablet_manager.cc:1905] T d44ff192dcc44035970773e483cc7e24 P 5cd7e7e149cf45f5a644b92416699061: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:12:30.639186 4368 ts_tablet_manager.cc:1905] T d44ff192dcc44035970773e483cc7e24 P e5491767f24d4ac5812d23cf91db8d0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:12:30.641597 4367 ts_tablet_manager.cc:1918] T d44ff192dcc44035970773e483cc7e24 P 5cd7e7e149cf45f5a644b92416699061: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:12:30.641748 4367 log.cc:1196] T d44ff192dcc44035970773e483cc7e24 P 5cd7e7e149cf45f5a644b92416699061: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/d44ff192dcc44035970773e483cc7e24
I20260807 15:12:30.642077 4367 ts_tablet_manager.cc:1939] T d44ff192dcc44035970773e483cc7e24 P 5cd7e7e149cf45f5a644b92416699061: Deleting consensus metadata
I20260807 15:12:30.642947 4368 ts_tablet_manager.cc:1918] T d44ff192dcc44035970773e483cc7e24 P e5491767f24d4ac5812d23cf91db8d0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:12:30.643069 4368 log.cc:1196] T d44ff192dcc44035970773e483cc7e24 P e5491767f24d4ac5812d23cf91db8d0a: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/d44ff192dcc44035970773e483cc7e24
I20260807 15:12:30.643508 4368 ts_tablet_manager.cc:1939] T d44ff192dcc44035970773e483cc7e24 P e5491767f24d4ac5812d23cf91db8d0a: Deleting consensus metadata
I20260807 15:12:30.643548 3563 catalog_manager.cc:4841] TS 5cd7e7e149cf45f5a644b92416699061 (127.15.226.66:39565): tablet d44ff192dcc44035970773e483cc7e24 (table test-table [id=5fcd8343f3a945fbb168f0c6436c7e36]) successfully deleted
I20260807 15:12:30.644924 3563 catalog_manager.cc:4841] TS e5491767f24d4ac5812d23cf91db8d0a (127.15.226.67:35353): tablet d44ff192dcc44035970773e483cc7e24 (table test-table [id=5fcd8343f3a945fbb168f0c6436c7e36]) successfully deleted
W20260807 15:12:30.646576 3569 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.226.65:40159 (error 108)
I20260807 15:12:30.646962 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 3890
I20260807 15:12:30.655570 4324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:30.655089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:40159 (local address 127.0.0.1:38196)
0807 15:12:30.655308 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:30.655434 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.65:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260807 15:12:30.655879 3569 catalog_manager.cc:4584] TS 53f7598c906449ddad9b0f69ca2278ef (127.15.226.65:40159): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d44ff192dcc44035970773e483cc7e24: Network error: Client connection negotiation failed: client connection to 127.15.226.65:40159: connect: Connection refused (error 111)
I20260807 15:12:30.658260 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 4098
I20260807 15:12:30.670883 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 3542
I20260807 15:12:30.687633 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:46183
--webserver_interface=127.15.226.126
--webserver_port=35357
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:46183
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:30.810084 4371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:30.810441 4371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:30.810487 4371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:30.814697 4371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:12:30.814831 4371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:30.814870 4371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:30.814900 4371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:12:30.814929 4371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:12:30.820708 4371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:46183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:46183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=35357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:30.821877 4371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:30.823036 4371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:12:30.839834 4371 server_base.cc:1034] running on GCE node
W20260807 15:12:30.839676 4383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:30.839675 4384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:30.839762 4386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:30.840912 4371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:30.842173 4371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:30.843355 4371 hybrid_clock.cc:648] HybridClock initialized: now 1786115550843326 us; error 46 us; skew 500 ppm
Aug 07 15:12:30 dist-test-slave-3kk6 krb5kdc[3522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115550, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:30.846616 4371 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:12:30.847716 4371 webserver.cc:460] Webserver started at http://127.15.226.126:35357/ using document root <none> and password file <none>
I20260807 15:12:30.848301 4371 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:30.848380 4371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:30.852764 4371 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:12:30.855254 4395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:30.856696 4371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 15:12:30.856848 4371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "fdc5c807613c4ce5a462f9b6644a8486"
format_stamp: "Formatted at 2026-08-07 15:12:29 on dist-test-slave-3kk6"
server_key: "5d05a3742c561cdf587ecccc3179aa9b"
server_key_iv: "7d2387efd6ab98088f5ced80d4eda109"
server_key_version: "encryptionkey@0"
I20260807 15:12:30.857549 4371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:30.880268 4371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:30.883896 4371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:30.884146 4371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:30.892300 4371 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:46183
I20260807 15:12:30.892613 4475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:46183 every 8 connection(s)
I20260807 15:12:30.893646 4371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:12:30.896584 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 4371
I20260807 15:12:30.897152 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:40159
--local_ip_for_outbound_sockets=127.15.226.65
--tserver_master_addrs=127.15.226.126:46183
--webserver_port=37363
--webserver_interface=127.15.226.65
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:12:30.898984 4477 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:12:30.900902 4477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486: Bootstrap starting.
I20260807 15:12:30.920281 4477 log.cc:824] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:30.930240 4477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:12:30.930822 4477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486: Bootstrap complete.
I20260807 15:12:30.935501 4477 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc5c807613c4ce5a462f9b6644a8486" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46183 } }
I20260807 15:12:30.935999 4477 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdc5c807613c4ce5a462f9b6644a8486, State: Initialized, Role: FOLLOWER
I20260807 15:12:30.936551 4477 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc5c807613c4ce5a462f9b6644a8486" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46183 } }
I20260807 15:12:30.936717 4477 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:12:30.936831 4477 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:12:30.936964 4477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:12:30.939393 4477 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc5c807613c4ce5a462f9b6644a8486" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46183 } }
I20260807 15:12:30.939817 4477 leader_election.cc:302] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdc5c807613c4ce5a462f9b6644a8486; no voters:
I20260807 15:12:30.940207 4477 leader_election.cc:288] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 15:12:30.940372 4484 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:12:30.940666 4484 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 2 LEADER]: Becoming Leader. State: Replica: fdc5c807613c4ce5a462f9b6644a8486, State: Running, Role: LEADER
I20260807 15:12:30.941215 4484 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc5c807613c4ce5a462f9b6644a8486" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46183 } }
I20260807 15:12:30.941691 4477 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:12:30.942860 4485 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fdc5c807613c4ce5a462f9b6644a8486" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc5c807613c4ce5a462f9b6644a8486" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46183 } } }
I20260807 15:12:30.943007 4485 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:30.943388 4486 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdc5c807613c4ce5a462f9b6644a8486. Latest consensus state: current_term: 2 leader_uuid: "fdc5c807613c4ce5a462f9b6644a8486" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc5c807613c4ce5a462f9b6644a8486" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46183 } } }
I20260807 15:12:30.943497 4486 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:30.947682 4504 catalog_manager.cc:1259] Loaded cluster ID: 970ee3cf882940cca581d7917f8e935c
I20260807 15:12:30.947794 4504 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486: loading cluster ID for follower catalog manager: success
W20260807 15:12:30.949241 4504 catalog_manager.cc:1576] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260807 15:12:30.949345 4504 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260807 15:12:30.949435 4494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:12:30.952109 4494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:12:30.952687 4494 catalog_manager.cc:1259] Loaded cluster ID: 970ee3cf882940cca581d7917f8e935c
I20260807 15:12:30.952759 4494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:12:30.953923 4494 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 15:12:30.958837 4494 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 15:12:30.965091 4494 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:12:30.966039 4494 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486: Loaded TSK: 0
I20260807 15:12:30.967373 4494 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:12:31.029178 4481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:31.029584 4481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:31.029673 4481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:31.033998 4481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:31.034137 4481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:31.034245 4481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:12:31.040027 4481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:40159
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=37363
--tserver_master_addrs=127.15.226.126:46183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:31.041205 4481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:31.042457 4481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:31.053998 4518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:31.054061 4517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:31.054561 4521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:31.054924 4481 server_base.cc:1034] running on GCE node
I20260807 15:12:31.055507 4481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:31.056289 4481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:31.057474 4481 hybrid_clock.cc:648] HybridClock initialized: now 1786115551057443 us; error 54 us; skew 500 ppm
Aug 07 15:12:31 dist-test-slave-3kk6 krb5kdc[3522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115551, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:31.060698 4481 init.cc:377] Logged in from keytab as kudu/127.15.226.65@KRBTEST.COM (short username kudu)
I20260807 15:12:31.061908 4481 webserver.cc:460] Webserver started at http://127.15.226.65:37363/ using document root <none> and password file <none>
I20260807 15:12:31.062599 4481 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:31.062773 4481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:31.067227 4481 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260807 15:12:31.069577 4536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:31.071048 4481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:12:31.071156 4481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "53f7598c906449ddad9b0f69ca2278ef"
format_stamp: "Formatted at 2026-08-07 15:12:29 on dist-test-slave-3kk6"
server_key: "f4a6173e00df66104e98268b61984ba7"
server_key_iv: "9d34c4a154e146af42cdc4f5b4023ad7"
server_key_version: "encryptionkey@0"
I20260807 15:12:31.071803 4481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:31.102924 4481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:31.106626 4481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:31.106904 4481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:31.107571 4481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:31.108749 4546 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 15:12:31.113288 4481 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 15:12:31.113399 4481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.001s
I20260807 15:12:31.113484 4481 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 15:12:31.115458 4481 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 15:12:31.115542 4481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260807 15:12:31.115850 4546 tablet_bootstrap.cc:492] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef: Bootstrap starting.
I20260807 15:12:31.127449 4481 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:40159
I20260807 15:12:31.127537 4707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:40159 every 8 connection(s)
I20260807 15:12:31.128584 4481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:12:31.137192 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 4481
I20260807 15:12:31.137765 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:39565
--local_ip_for_outbound_sockets=127.15.226.66
--tserver_master_addrs=127.15.226.126:46183
--webserver_port=34541
--webserver_interface=127.15.226.66
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:12:31.137784 4546 log.cc:824] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef: Log is configured to *not* fsync() on all Append() calls
Aug 07 15:12:31 dist-test-slave-3kk6 krb5kdc[3522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115551, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:12:31.147716 4546 tablet_bootstrap.cc:492] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:12:31.148272 4546 tablet_bootstrap.cc:492] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef: Bootstrap complete.
I20260807 15:12:31.150003 4546 ts_tablet_manager.cc:1397] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef: Time spent bootstrapping tablet: real 0.034s user 0.026s sys 0.005s
I20260807 15:12:31.150409 4713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:31.131052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:53473 (local address 127.15.226.126:46183)
0807 15:12:31.131517 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:31.131542 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:31.131990 (+ 448us) server_negotiation.cc:407] Connection header received
0807 15:12:31.133125 (+ 1135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:31.133144 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:31.133658 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:31.134120 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:31.135484 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.137020 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.139006 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.139738 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.139928 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:31.143638 (+ 3710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:31.143647 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:31.143665 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:31.143708 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:31.146118 (+ 2410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:31.146891 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:31.146898 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:31.146903 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:31.147073 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:31.147541 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:31.147544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:31.147546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:31.148063 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:31.148198 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:31.148968 (+ 770us) server_negotiation.cc:299] Negotiation successful
0807 15:12:31.149346 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":139,"threads_started":1}
I20260807 15:12:31.151579 4714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:31.131009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.65:53473)
0807 15:12:31.131632 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:31.131706 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:31.132640 (+ 934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:31.134215 (+ 1575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:31.134222 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:31.134632 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:31.135260 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:31.135278 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.137246 (+ 1968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.137252 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:31.138775 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:31.138794 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.139998 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.140004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:31.140243 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:31.141086 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:31.141109 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:31.143310 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:31.146311 (+ 3001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:31.146317 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:31.146330 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:31.146737 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:31.147203 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:31.147206 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:31.147208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:31.147414 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:31.148290 (+ 876us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:31.148298 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:31.148600 (+ 302us) client_negotiation.cc:769] Sending connection context
0807 15:12:31.150230 (+ 1630us) client_negotiation.cc:240] Negotiation successful
0807 15:12:31.150680 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":159,"threads_started":1}
I20260807 15:12:31.153127 4546 raft_consensus.cc:348] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } }
I20260807 15:12:31.153690 4546 raft_consensus.cc:729] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53f7598c906449ddad9b0f69ca2278ef, State: Initialized, Role: FOLLOWER
I20260807 15:12:31.154160 4546 consensus_queue.cc:260] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } }
I20260807 15:12:31.154361 4708 heartbeater.cc:347] Connected to a master server at 127.15.226.126:46183
I20260807 15:12:31.154608 4708 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:31.155220 4708 heartbeater.cc:511] Master 127.15.226.126:46183 requested a full tablet report, sending...
I20260807 15:12:31.155226 4546 ts_tablet_manager.cc:1428] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
I20260807 15:12:31.157958 4421 ts_manager.cc:194] Registered new tserver with Master: 53f7598c906449ddad9b0f69ca2278ef (127.15.226.65:40159)
I20260807 15:12:31.161865 4421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.65@KRBTEST.COM'} at 127.15.226.65:53473
I20260807 15:12:31.164990 4708 heartbeater.cc:503] Master 127.15.226.126:46183 was elected leader, sending a full tablet report...
Aug 07 15:12:31 dist-test-slave-3kk6 krb5kdc[3522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115550, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for kudu/127.15.226.65@KRBTEST.COM
I20260807 15:12:31.176534 4719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:31.161291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:40159 (local address 127.0.0.1:38208)
0807 15:12:31.162248 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:31.162306 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:31.162498 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:31.165247 (+ 2749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:31.165263 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:31.165769 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:31.165994 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:31.166005 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.167500 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.167503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:31.168185 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:31.168196 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.168996 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.168999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:31.169105 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:31.169851 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:31.169863 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:31.171578 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:31.174312 (+ 2734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:31.174324 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:31.174328 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:31.174611 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:31.175114 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:31.175116 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:31.175118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:31.175212 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:31.175825 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:31.175831 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:31.176013 (+ 182us) client_negotiation.cc:769] Sending connection context
0807 15:12:31.176184 (+ 171us) client_negotiation.cc:240] Negotiation successful
0807 15:12:31.176382 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"thread_start_us":142,"threads_started":1}
I20260807 15:12:31.176651 4720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:31.161405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38208 (local address 127.15.226.65:40159)
0807 15:12:31.161888 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:31.161902 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:31.162432 (+ 530us) server_negotiation.cc:407] Connection header received
0807 15:12:31.162702 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:31.162708 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:31.162975 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:31.163147 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:31.166151 (+ 3004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.167337 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.168317 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.168835 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.168954 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:31.171805 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:31.171819 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:31.171833 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:31.171870 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:31.174162 (+ 2292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:31.174893 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:31.174900 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:31.174902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:31.175007 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:31.175310 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:31.175313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:31.175314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:31.175653 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:31.175789 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:31.176276 (+ 487us) server_negotiation.cc:299] Negotiation successful
0807 15:12:31.176479 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":124,"threads_started":1}
I20260807 15:12:31.177840 4606 tablet_service.cc:1514] Processing DeleteTablet for tablet d44ff192dcc44035970773e483cc7e24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:12:30 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:38208
I20260807 15:12:31.179046 4721 tablet_replica.cc:330] stopping tablet replica
I20260807 15:12:31.179426 4721 raft_consensus.cc:2229] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:12:31.179715 4721 raft_consensus.cc:2258] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:12:31.180789 4721 ts_tablet_manager.cc:1905] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:12:31.185022 4721 ts_tablet_manager.cc:1918] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:12:31.185165 4721 log.cc:1196] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/wal/wals/d44ff192dcc44035970773e483cc7e24
I20260807 15:12:31.185474 4721 ts_tablet_manager.cc:1939] T d44ff192dcc44035970773e483cc7e24 P 53f7598c906449ddad9b0f69ca2278ef: Deleting consensus metadata
I20260807 15:12:31.187461 4403 catalog_manager.cc:4841] TS 53f7598c906449ddad9b0f69ca2278ef (127.15.226.65:40159): tablet d44ff192dcc44035970773e483cc7e24 (table test-table [id=5fcd8343f3a945fbb168f0c6436c7e36]) successfully deleted
W20260807 15:12:31.270507 4716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:31.270926 4716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:31.270982 4716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:31.275550 4716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:31.275640 4716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:31.275736 4716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:12:31.281706 4716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:39565
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=34541
--tserver_master_addrs=127.15.226.126:46183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:31.283063 4716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:31.284212 4716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:31.295511 4730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:31.295514 4727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:31.295511 4728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:31.295774 4716 server_base.cc:1034] running on GCE node
I20260807 15:12:31.296525 4716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:31.297334 4716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:31.298513 4716 hybrid_clock.cc:648] HybridClock initialized: now 1786115551298497 us; error 34 us; skew 500 ppm
Aug 07 15:12:31 dist-test-slave-3kk6 krb5kdc[3522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115551, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:31.301879 4716 init.cc:377] Logged in from keytab as kudu/127.15.226.66@KRBTEST.COM (short username kudu)
I20260807 15:12:31.303148 4716 webserver.cc:460] Webserver started at http://127.15.226.66:34541/ using document root <none> and password file <none>
I20260807 15:12:31.303788 4716 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:31.303915 4716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:31.308681 4716 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:12:31.311257 4744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:31.312834 4716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:12:31.313025 4716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "5cd7e7e149cf45f5a644b92416699061"
format_stamp: "Formatted at 2026-08-07 15:12:29 on dist-test-slave-3kk6"
server_key: "c768243eab5c25f22a4c992a18fa5735"
server_key_iv: "d81becac286a08a3a5dcf37358603962"
server_key_version: "encryptionkey@0"
I20260807 15:12:31.313762 4716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:31.328169 4716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:31.331956 4716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:31.332206 4716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:31.332904 4716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:31.333869 4716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:31.333945 4716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:31.334022 4716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:31.334064 4716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:31.344810 4716 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:39565
I20260807 15:12:31.344848 4919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:39565 every 8 connection(s)
I20260807 15:12:31.345992 4716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:12:31.347827 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 4716
I20260807 15:12:31.348374 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:35353
--local_ip_for_outbound_sockets=127.15.226.67
--tserver_master_addrs=127.15.226.126:46183
--webserver_port=40823
--webserver_interface=127.15.226.67
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:12:31 dist-test-slave-3kk6 krb5kdc[3522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115551, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:12:31.364530 4713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:31.348194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:42267 (local address 127.15.226.126:46183)
0807 15:12:31.348357 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:31.348361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:31.349560 (+ 1199us) server_negotiation.cc:407] Connection header received
0807 15:12:31.351237 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:31.351242 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:31.351310 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:31.351423 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:31.352840 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.353710 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.355400 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.355861 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.356023 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:31.359569 (+ 3546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:31.359576 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:31.359581 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:31.359630 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:31.361670 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:31.362462 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:31.362466 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:31.362468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:31.362587 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:31.363091 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:31.363098 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:31.363102 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:31.363411 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:31.363543 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:31.364179 (+ 636us) server_negotiation.cc:299] Negotiation successful
0807 15:12:31.364339 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:12:31.365548 4924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:31.348751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.66:42267)
0807 15:12:31.349342 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:31.349395 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:31.350969 (+ 1574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:31.351642 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:31.351650 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:31.352100 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:31.352652 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:31.352667 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.353924 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.353928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:31.355231 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:31.355245 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.356091 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.356094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:31.356273 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:31.357076 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:31.357096 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:31.359290 (+ 2194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:31.361886 (+ 2596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:31.361893 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:31.361911 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:31.362302 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:31.362784 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:31.362786 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:31.362788 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:31.362959 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:31.363558 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:31.363565 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:31.363876 (+ 311us) client_negotiation.cc:769] Sending connection context
0807 15:12:31.364066 (+ 190us) client_negotiation.cc:240] Negotiation successful
0807 15:12:31.364384 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":160,"threads_started":1}
I20260807 15:12:31.367043 4920 heartbeater.cc:347] Connected to a master server at 127.15.226.126:46183
I20260807 15:12:31.367341 4920 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:31.368007 4920 heartbeater.cc:511] Master 127.15.226.126:46183 requested a full tablet report, sending...
I20260807 15:12:31.369336 4421 ts_manager.cc:194] Registered new tserver with Master: 5cd7e7e149cf45f5a644b92416699061 (127.15.226.66:39565)
I20260807 15:12:31.369979 4421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.66@KRBTEST.COM'} at 127.15.226.66:42267
W20260807 15:12:31.480047 4925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:31.480430 4925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:31.480516 4925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:31.484912 4925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:31.485005 4925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:31.485101 4925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:12:31.491132 4925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:35353
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=40823
--tserver_master_addrs=127.15.226.126:46183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:31.492394 4925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:31.493444 4925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:31.504844 4932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:31.504804 4931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:31.505044 4925 server_base.cc:1034] running on GCE node
W20260807 15:12:31.504833 4935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:31.505815 4925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:31.506587 4925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:31.507834 4925 hybrid_clock.cc:648] HybridClock initialized: now 1786115551507814 us; error 40 us; skew 500 ppm
Aug 07 15:12:31 dist-test-slave-3kk6 krb5kdc[3522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115551, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:31.511242 4925 init.cc:377] Logged in from keytab as kudu/127.15.226.67@KRBTEST.COM (short username kudu)
I20260807 15:12:31.512358 4925 webserver.cc:460] Webserver started at http://127.15.226.67:40823/ using document root <none> and password file <none>
I20260807 15:12:31.513025 4925 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:31.513156 4925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:31.517892 4925 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:12:31.520642 4946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:31.522317 4925 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 15:12:31.522508 4925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "e5491767f24d4ac5812d23cf91db8d0a"
format_stamp: "Formatted at 2026-08-07 15:12:30 on dist-test-slave-3kk6"
server_key: "2c6a399c214f7bb4cb9bd04690b07931"
server_key_iv: "f10174211d2c301d86b53729415a44bc"
server_key_version: "encryptionkey@0"
I20260807 15:12:31.523267 4925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:31.574074 4925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:31.578413 4925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:31.578752 4925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:31.579617 4925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:31.580662 4925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:31.580726 4925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:31.580813 4925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:31.580856 4925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:31.592813 4925 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:35353
I20260807 15:12:31.592823 5115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:35353 every 8 connection(s)
I20260807 15:12:31.594105 4925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:12:31.600221 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 4925
Aug 07 15:12:31 dist-test-slave-3kk6 krb5kdc[3522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115551, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:12:31.610532 5119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:31.601121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44306 (local address 127.15.226.126:46183)
0807 15:12:31.601424 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:31.601429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:31.601619 (+ 190us) server_negotiation.cc:407] Connection header received
0807 15:12:31.601805 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:31.601807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:31.601852 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:31.601954 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:31.602811 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.603360 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.604276 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.604518 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.604633 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:31.606380 (+ 1747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:31.606384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:31.606386 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:31.606418 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:31.608505 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:31.609009 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:31.609016 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:31.609020 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:31.609149 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:31.609505 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:31.609509 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:31.609511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:31.609784 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:31.609999 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:31.610230 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 15:12:31.610383 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":113,"threads_started":1}
I20260807 15:12:31.612079 4713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:31.596336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:41389 (local address 127.15.226.126:46183)
0807 15:12:31.596498 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:31.596501 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:31.597655 (+ 1154us) server_negotiation.cc:407] Connection header received
0807 15:12:31.599250 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:31.599253 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:31.599318 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:31.599411 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:31.601161 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.601869 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.603480 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.603742 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.603873 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:31.607202 (+ 3329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:31.607206 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:31.607208 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:31.607248 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:31.609028 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:31.609730 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:31.609733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:31.609734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:31.609847 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:31.610322 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:31.610328 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:31.610332 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:31.610819 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:31.610934 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:31.611740 (+ 806us) server_negotiation.cc:299] Negotiation successful
0807 15:12:31.611896 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 15:12:31.613221 5118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:31.596770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.67:41389)
0807 15:12:31.597454 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:31.597507 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:31.598997 (+ 1490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:31.599628 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:31.599636 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:31.600212 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:31.600938 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:31.600957 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.602049 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.602059 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:31.603326 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:31.603340 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.603941 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.603946 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:31.604152 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:31.604912 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:31.604933 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:31.607014 (+ 2081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:31.609166 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:31.609175 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:31.609200 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:31.609602 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:31.609963 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:31.609966 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:31.609968 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:31.610186 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:31.610977 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:31.610985 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:31.611367 (+ 382us) client_negotiation.cc:769] Sending connection context
0807 15:12:31.611614 (+ 247us) client_negotiation.cc:240] Negotiation successful
0807 15:12:31.612047 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":223,"threads_started":1}
I20260807 15:12:31.614523 5116 heartbeater.cc:347] Connected to a master server at 127.15.226.126:46183
I20260807 15:12:31.614840 5116 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:31.615437 5116 heartbeater.cc:511] Master 127.15.226.126:46183 requested a full tablet report, sending...
I20260807 15:12:31.616564 4421 ts_manager.cc:194] Registered new tserver with Master: e5491767f24d4ac5812d23cf91db8d0a (127.15.226.67:35353)
I20260807 15:12:31.617203 4421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.67@KRBTEST.COM'} at 127.15.226.67:41389
I20260807 15:12:31.625164 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:12:31.630122 4713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:31.625932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44316 (local address 127.15.226.126:46183)
0807 15:12:31.626103 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:31.626108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:31.626366 (+ 258us) server_negotiation.cc:407] Connection header received
0807 15:12:31.626520 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:31.626522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:31.626640 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:31.626799 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:12:31.627085 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.627622 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.628596 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.628816 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.628902 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:31.629217 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:12:31.629681 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:12:31.629816 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:31.629874 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 15:12:31.629932 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 15:12:31.631018 4421 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:44316:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:12:31.634418 4421 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:12:31.643198 4606 tablet_service.cc:1467] Processing CreateTablet for tablet 62efbed6bd224fd0bca0fede08358f6f (DEFAULT_TABLE table=test-table [id=a5f90934cb174e70b69eaaa895ec0e2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:12:31.643961 4606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62efbed6bd224fd0bca0fede08358f6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:31.647045 4546 tablet_bootstrap.cc:492] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef: Bootstrap starting.
I20260807 15:12:31.648137 4546 tablet_bootstrap.cc:654] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:31.649233 5132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:31.642454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56668 (local address 127.15.226.66:39565)
0807 15:12:31.643019 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:31.643035 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:31.643061 (+ 26us) server_negotiation.cc:407] Connection header received
0807 15:12:31.643209 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:31.643214 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:31.643428 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:31.643597 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:12:31.644176 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.645373 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.647282 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.648217 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.648328 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:31.648500 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:31.648964 (+ 464us) server_negotiation.cc:299] Negotiation successful
0807 15:12:31.649080 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":149,"threads_started":1}
I20260807 15:12:31.649232 4719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:31.641972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:35353 (local address 127.0.0.1:37574)
0807 15:12:31.642145 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:31.642164 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:31.642271 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:31.643664 (+ 1393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:31.643668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:31.643685 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:12:31.643791 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:31.643797 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.645512 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.645515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:31.647240 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:31.647250 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.648644 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.648649 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:31.648821 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:31.648849 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 15:12:31.648937 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 15:12:31.649043 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":279}
I20260807 15:12:31.649209 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:31.642256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:39565 (local address 127.0.0.1:56668)
0807 15:12:31.642950 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:31.642966 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:31.643065 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:31.643788 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:31.643790 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:31.643804 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:12:31.643946 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:31.643953 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.645526 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.645531 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:31.647118 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:31.647135 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.648575 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.648580 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:31.648756 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:31.648816 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 15:12:31.648909 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 15:12:31.649002 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"mutex_wait_us":103,"thread_start_us":125,"threads_started":1}
I20260807 15:12:31.649597 5129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:31.642465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37574 (local address 127.15.226.67:35353)
0807 15:12:31.643102 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:31.643114 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:31.643138 (+ 24us) server_negotiation.cc:407] Connection header received
0807 15:12:31.643218 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:31.643225 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:31.643449 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:31.643661 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:12:31.643935 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.645258 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.647371 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.648506 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.648621 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:31.648778 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:31.649312 (+ 534us) server_negotiation.cc:299] Negotiation successful
0807 15:12:31.649442 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":119,"threads_started":1}
I20260807 15:12:31.649770 4546 tablet_bootstrap.cc:492] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef: No bootstrap required, opened a new log
I20260807 15:12:31.649901 4546 ts_tablet_manager.cc:1397] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:12:31.650345 4546 raft_consensus.cc:348] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } }
I20260807 15:12:31.650480 4546 raft_consensus.cc:374] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:31.650535 4546 raft_consensus.cc:729] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53f7598c906449ddad9b0f69ca2278ef, State: Initialized, Role: FOLLOWER
I20260807 15:12:31.650719 4546 consensus_queue.cc:260] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } }
I20260807 15:12:31.651067 4546 ts_tablet_manager.cc:1428] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 15:12:31.653039 4816 tablet_service.cc:1467] Processing CreateTablet for tablet 62efbed6bd224fd0bca0fede08358f6f (DEFAULT_TABLE table=test-table [id=a5f90934cb174e70b69eaaa895ec0e2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:12:31.654273 4816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62efbed6bd224fd0bca0fede08358f6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:31.653837 5023 tablet_service.cc:1467] Processing CreateTablet for tablet 62efbed6bd224fd0bca0fede08358f6f (DEFAULT_TABLE table=test-table [id=a5f90934cb174e70b69eaaa895ec0e2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:12:31.655131 5023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62efbed6bd224fd0bca0fede08358f6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:31.662477 5137 tablet_bootstrap.cc:492] T 62efbed6bd224fd0bca0fede08358f6f P e5491767f24d4ac5812d23cf91db8d0a: Bootstrap starting.
I20260807 15:12:31.662468 5135 tablet_bootstrap.cc:492] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061: Bootstrap starting.
I20260807 15:12:31.665150 5137 tablet_bootstrap.cc:654] T 62efbed6bd224fd0bca0fede08358f6f P e5491767f24d4ac5812d23cf91db8d0a: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:31.665582 5135 tablet_bootstrap.cc:654] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:31.666415 5137 log.cc:824] T 62efbed6bd224fd0bca0fede08358f6f P e5491767f24d4ac5812d23cf91db8d0a: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:31.666415 5135 log.cc:824] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:31.668499 5135 tablet_bootstrap.cc:492] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061: No bootstrap required, opened a new log
I20260807 15:12:31.668511 5137 tablet_bootstrap.cc:492] T 62efbed6bd224fd0bca0fede08358f6f P e5491767f24d4ac5812d23cf91db8d0a: No bootstrap required, opened a new log
I20260807 15:12:31.668730 5135 ts_tablet_manager.cc:1397] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 15:12:31.668746 5137 ts_tablet_manager.cc:1397] T 62efbed6bd224fd0bca0fede08358f6f P e5491767f24d4ac5812d23cf91db8d0a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 15:12:31.672400 5137 raft_consensus.cc:348] T 62efbed6bd224fd0bca0fede08358f6f P e5491767f24d4ac5812d23cf91db8d0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } }
I20260807 15:12:31.672400 5135 raft_consensus.cc:348] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } }
I20260807 15:12:31.672734 5137 raft_consensus.cc:374] T 62efbed6bd224fd0bca0fede08358f6f P e5491767f24d4ac5812d23cf91db8d0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:31.672730 5135 raft_consensus.cc:374] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:31.672811 5137 raft_consensus.cc:729] T 62efbed6bd224fd0bca0fede08358f6f P e5491767f24d4ac5812d23cf91db8d0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5491767f24d4ac5812d23cf91db8d0a, State: Initialized, Role: FOLLOWER
I20260807 15:12:31.672833 5135 raft_consensus.cc:729] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cd7e7e149cf45f5a644b92416699061, State: Initialized, Role: FOLLOWER
I20260807 15:12:31.673318 5137 consensus_queue.cc:260] T 62efbed6bd224fd0bca0fede08358f6f P e5491767f24d4ac5812d23cf91db8d0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } }
I20260807 15:12:31.673318 5135 consensus_queue.cc:260] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } }
I20260807 15:12:31.674118 5116 heartbeater.cc:503] Master 127.15.226.126:46183 was elected leader, sending a full tablet report...
I20260807 15:12:31.674484 5137 ts_tablet_manager.cc:1428] T 62efbed6bd224fd0bca0fede08358f6f P e5491767f24d4ac5812d23cf91db8d0a: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260807 15:12:31.674505 5135 ts_tablet_manager.cc:1428] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 15:12:31.674841 4920 heartbeater.cc:503] Master 127.15.226.126:46183 was elected leader, sending a full tablet report...
I20260807 15:12:31.740273 5142 raft_consensus.cc:482] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:12:31.740556 5142 raft_consensus.cc:504] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } }
I20260807 15:12:31.741936 5142 leader_election.cc:288] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5491767f24d4ac5812d23cf91db8d0a (127.15.226.67:35353), 53f7598c906449ddad9b0f69ca2278ef (127.15.226.65:40159)
I20260807 15:12:31.746510 5129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:31.742336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:49643 (local address 127.15.226.67:35353)
0807 15:12:31.742475 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:31.742480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:31.742576 (+ 96us) server_negotiation.cc:407] Connection header received
0807 15:12:31.742784 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:31.742790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:31.742868 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:31.742992 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:12:31.743459 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.744002 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.745206 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.745815 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.745957 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:31.746017 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:31.746269 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 15:12:31.746371 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 15:12:31.746512 4924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:31.742216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:35353 (local address 127.15.226.66:49643)
0807 15:12:31.742452 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:31.742472 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:31.742595 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:31.743067 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:31.743071 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:31.743099 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:12:31.743286 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:31.743294 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.744146 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.744149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:31.745025 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:31.745040 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.745977 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.745984 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:31.746113 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:31.746139 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 15:12:31.746227 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 15:12:31.746347 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":7}
I20260807 15:12:31.746593 5143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:31.742273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:40159 (local address 127.15.226.66:41039)
0807 15:12:31.742714 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:31.742729 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:31.742808 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:31.743136 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:31.743138 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:31.743151 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:12:31.743298 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:31.743304 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.744234 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.744237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:31.745048 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:31.745058 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.746110 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.746116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:31.746255 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:31.746284 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 15:12:31.746374 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 15:12:31.746447 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":23,"thread_start_us":130,"threads_started":1}
I20260807 15:12:31.747046 5146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:31.742455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:41039 (local address 127.15.226.65:40159)
0807 15:12:31.742774 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:31.742779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:31.742795 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:12:31.742888 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:31.742893 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:31.742993 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:31.743100 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:12:31.743458 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.744022 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.745240 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.745914 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.746051 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:31.746251 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:31.746789 (+ 538us) server_negotiation.cc:299] Negotiation successful
0807 15:12:31.746922 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":109,"threads_started":1}
I20260807 15:12:31.747617 5048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62efbed6bd224fd0bca0fede08358f6f" candidate_uuid: "5cd7e7e149cf45f5a644b92416699061" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5491767f24d4ac5812d23cf91db8d0a" is_pre_election: true
I20260807 15:12:31.747617 4639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62efbed6bd224fd0bca0fede08358f6f" candidate_uuid: "5cd7e7e149cf45f5a644b92416699061" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53f7598c906449ddad9b0f69ca2278ef" is_pre_election: true
I20260807 15:12:31.747975 5048 raft_consensus.cc:2454] T 62efbed6bd224fd0bca0fede08358f6f P e5491767f24d4ac5812d23cf91db8d0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd7e7e149cf45f5a644b92416699061 in term 0.
I20260807 15:12:31.747977 4639 raft_consensus.cc:2454] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cd7e7e149cf45f5a644b92416699061 in term 0.
I20260807 15:12:31.748585 4752 leader_election.cc:302] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 53f7598c906449ddad9b0f69ca2278ef, 5cd7e7e149cf45f5a644b92416699061, e5491767f24d4ac5812d23cf91db8d0a; no voters:
I20260807 15:12:31.748953 5142 raft_consensus.cc:2781] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:12:31.749060 5142 raft_consensus.cc:482] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:12:31.749127 5142 raft_consensus.cc:3037] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:31.750329 5142 raft_consensus.cc:504] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } }
I20260807 15:12:31.750844 5142 leader_election.cc:288] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061 [CANDIDATE]: Term 1 election: Requested vote from peers e5491767f24d4ac5812d23cf91db8d0a (127.15.226.67:35353), 53f7598c906449ddad9b0f69ca2278ef (127.15.226.65:40159)
I20260807 15:12:31.751339 5048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62efbed6bd224fd0bca0fede08358f6f" candidate_uuid: "5cd7e7e149cf45f5a644b92416699061" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5491767f24d4ac5812d23cf91db8d0a"
I20260807 15:12:31.751338 4639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62efbed6bd224fd0bca0fede08358f6f" candidate_uuid: "5cd7e7e149cf45f5a644b92416699061" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53f7598c906449ddad9b0f69ca2278ef"
I20260807 15:12:31.751497 4639 raft_consensus.cc:3037] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:31.751497 5048 raft_consensus.cc:3037] T 62efbed6bd224fd0bca0fede08358f6f P e5491767f24d4ac5812d23cf91db8d0a [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:31.752985 5048 raft_consensus.cc:2454] T 62efbed6bd224fd0bca0fede08358f6f P e5491767f24d4ac5812d23cf91db8d0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd7e7e149cf45f5a644b92416699061 in term 1.
I20260807 15:12:31.752985 4639 raft_consensus.cc:2454] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cd7e7e149cf45f5a644b92416699061 in term 1.
I20260807 15:12:31.753348 4752 leader_election.cc:302] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53f7598c906449ddad9b0f69ca2278ef, 5cd7e7e149cf45f5a644b92416699061; no voters:
I20260807 15:12:31.753588 5142 raft_consensus.cc:2781] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:12:31.753871 5142 raft_consensus.cc:686] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061 [term 1 LEADER]: Becoming Leader. State: Replica: 5cd7e7e149cf45f5a644b92416699061, State: Running, Role: LEADER
I20260807 15:12:31.754253 5142 consensus_queue.cc:237] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } }
I20260807 15:12:31.757593 4421 catalog_manager.cc:5495] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cd7e7e149cf45f5a644b92416699061 (127.15.226.66). New cstate: current_term: 1 leader_uuid: "5cd7e7e149cf45f5a644b92416699061" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } health_report { overall_health: UNKNOWN } } }
I20260807 15:12:31.780431 5132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:31.776569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56674 (local address 127.15.226.66:39565)
0807 15:12:31.776812 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:31.776816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:31.776828 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:12:31.776909 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:31.776911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:31.777004 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:31.777149 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:12:31.777421 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.777939 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.778839 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:31.779057 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:31.779168 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:31.779525 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:12:31.779985 (+ 460us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:12:31.780132 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:31.780183 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 15:12:31.780246 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 15:12:31.787278 5048 raft_consensus.cc:1261] T 62efbed6bd224fd0bca0fede08358f6f P e5491767f24d4ac5812d23cf91db8d0a [term 1 FOLLOWER]: Refusing update from remote peer 5cd7e7e149cf45f5a644b92416699061: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:12:31.787310 4639 raft_consensus.cc:1261] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Refusing update from remote peer 5cd7e7e149cf45f5a644b92416699061: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:12:31.788117 5147 consensus_queue.cc:1035] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:12:31.788349 5142 consensus_queue.cc:1035] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:12:31.794212 5151 mvcc.cc:205] Tried to move back new op lower bound from 7315929300110979072 to 7315929299988537344. Current Snapshot: MvccSnapshot[applied={T|T < 7315929300110979072}]
I20260807 15:12:31.794773 5153 mvcc.cc:205] Tried to move back new op lower bound from 7315929300110979072 to 7315929299988537344. Current Snapshot: MvccSnapshot[applied={T|T < 7315929300110979072}]
I20260807 15:12:31.804956 4421 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:44316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:12:31.805162 4421 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:44316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:12:31.807876 4421 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486: Sending DeleteTablet for 3 replicas of tablet 62efbed6bd224fd0bca0fede08358f6f
I20260807 15:12:31.808907 5023 tablet_service.cc:1514] Processing DeleteTablet for tablet 62efbed6bd224fd0bca0fede08358f6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:12:31 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:37574
I20260807 15:12:31.808911 4816 tablet_service.cc:1514] Processing DeleteTablet for tablet 62efbed6bd224fd0bca0fede08358f6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:12:31 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:56668
I20260807 15:12:31.809535 5165 tablet_replica.cc:330] T 62efbed6bd224fd0bca0fede08358f6f P e5491767f24d4ac5812d23cf91db8d0a: stopping tablet replica
I20260807 15:12:31.809496 4606 tablet_service.cc:1514] Processing DeleteTablet for tablet 62efbed6bd224fd0bca0fede08358f6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:12:31 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:38208
I20260807 15:12:31.809933 5170 tablet_replica.cc:330] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef: stopping tablet replica
I20260807 15:12:31.809942 5165 raft_consensus.cc:2229] T 62efbed6bd224fd0bca0fede08358f6f P e5491767f24d4ac5812d23cf91db8d0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:12:31.810135 5170 raft_consensus.cc:2229] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:12:31.810164 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 4481
I20260807 15:12:31.810338 5165 raft_consensus.cc:2258] T 62efbed6bd224fd0bca0fede08358f6f P e5491767f24d4ac5812d23cf91db8d0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:12:31.810384 5170 raft_consensus.cc:2258] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:12:31.810814 5168 tablet_replica.cc:330] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061: stopping tablet replica
I20260807 15:12:31.811239 5168 raft_consensus.cc:2229] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:12:31.811738 5168 raft_consensus.cc:2258] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:12:31.814338 5168 ts_tablet_manager.cc:1905] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:12:31.819283 5168 ts_tablet_manager.cc:1918] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:12:31.819492 5168 log.cc:1196] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/62efbed6bd224fd0bca0fede08358f6f
I20260807 15:12:31.820068 5168 ts_tablet_manager.cc:1939] T 62efbed6bd224fd0bca0fede08358f6f P 5cd7e7e149cf45f5a644b92416699061: Deleting consensus metadata
I20260807 15:12:31.822037 4399 catalog_manager.cc:4841] TS 5cd7e7e149cf45f5a644b92416699061 (127.15.226.66:39565): tablet 62efbed6bd224fd0bca0fede08358f6f (table test-table [id=a5f90934cb174e70b69eaaa895ec0e2a]) successfully deleted
I20260807 15:12:31.825098 5165 ts_tablet_manager.cc:1905] T 62efbed6bd224fd0bca0fede08358f6f P e5491767f24d4ac5812d23cf91db8d0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:12:31.828284 5165 ts_tablet_manager.cc:1918] T 62efbed6bd224fd0bca0fede08358f6f P e5491767f24d4ac5812d23cf91db8d0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:12:31.828434 5165 log.cc:1196] T 62efbed6bd224fd0bca0fede08358f6f P e5491767f24d4ac5812d23cf91db8d0a: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/62efbed6bd224fd0bca0fede08358f6f
I20260807 15:12:31.828804 5165 ts_tablet_manager.cc:1939] T 62efbed6bd224fd0bca0fede08358f6f P e5491767f24d4ac5812d23cf91db8d0a: Deleting consensus metadata
I20260807 15:12:31.830123 4399 catalog_manager.cc:4841] TS e5491767f24d4ac5812d23cf91db8d0a (127.15.226.67:35353): tablet 62efbed6bd224fd0bca0fede08358f6f (table test-table [id=a5f90934cb174e70b69eaaa895ec0e2a]) successfully deleted
W20260807 15:12:31.834944 4403 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.226.65:40159 (error 108)
I20260807 15:12:31.835666 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 4716
I20260807 15:12:31.836278 5128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:31.835878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:40159 (local address 127.0.0.1:38220)
0807 15:12:31.836094 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:31.836170 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.65:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260807 15:12:31.839600 4403 catalog_manager.cc:4584] TS 53f7598c906449ddad9b0f69ca2278ef (127.15.226.65:40159): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 62efbed6bd224fd0bca0fede08358f6f: Network error: Client connection negotiation failed: client connection to 127.15.226.65:40159: connect: Connection refused (error 111)
I20260807 15:12:31.849925 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 4925
I20260807 15:12:31.864675 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 4371
I20260807 15:12:31.879084 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:46183
--webserver_interface=127.15.226.126
--webserver_port=35357
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:46183
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:32.000653 5172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:32.001032 5172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:32.001117 5172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:32.005322 5172 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:12:32.005446 5172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:32.005486 5172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:32.005527 5172 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:12:32.005568 5172 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:12:32.011389 5172 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:46183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:46183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=35357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:32.012645 5172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:32.013811 5172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:12:32.023288 5172 server_base.cc:1034] running on GCE node
W20260807 15:12:32.023288 5184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:32.023494 5182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:32.023303 5179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:32.024230 5172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:32.025489 5172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:32.026731 5172 hybrid_clock.cc:648] HybridClock initialized: now 1786115552026713 us; error 32 us; skew 500 ppm
Aug 07 15:12:32 dist-test-slave-3kk6 krb5kdc[3522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115552, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:32.029932 5172 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:12:32.031064 5172 webserver.cc:460] Webserver started at http://127.15.226.126:35357/ using document root <none> and password file <none>
I20260807 15:12:32.031674 5172 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:32.031755 5172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:32.035903 5172 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:12:32.038280 5193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:32.039707 5172 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:12:32.039883 5172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "fdc5c807613c4ce5a462f9b6644a8486"
format_stamp: "Formatted at 2026-08-07 15:12:29 on dist-test-slave-3kk6"
server_key: "5d05a3742c561cdf587ecccc3179aa9b"
server_key_iv: "7d2387efd6ab98088f5ced80d4eda109"
server_key_version: "encryptionkey@0"
I20260807 15:12:32.040510 5172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:32.053061 5172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:32.056348 5172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:32.056591 5172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:32.064829 5172 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:46183
I20260807 15:12:32.065142 5266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:46183 every 8 connection(s)
I20260807 15:12:32.066148 5172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:12:32.067262 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 5172
I20260807 15:12:32.067746 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:40159
--local_ip_for_outbound_sockets=127.15.226.65
--tserver_master_addrs=127.15.226.126:46183
--webserver_port=37363
--webserver_interface=127.15.226.65
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:12:32.071115 5267 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:12:32.072943 5267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486: Bootstrap starting.
I20260807 15:12:32.092655 5267 log.cc:824] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:32.109299 5267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:12:32.109812 5267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486: Bootstrap complete.
I20260807 15:12:32.114559 5267 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc5c807613c4ce5a462f9b6644a8486" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46183 } }
I20260807 15:12:32.115164 5267 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdc5c807613c4ce5a462f9b6644a8486, State: Initialized, Role: FOLLOWER
I20260807 15:12:32.115711 5267 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc5c807613c4ce5a462f9b6644a8486" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46183 } }
I20260807 15:12:32.115880 5267 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:12:32.115978 5267 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:12:32.116113 5267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:12:32.118333 5267 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc5c807613c4ce5a462f9b6644a8486" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46183 } }
I20260807 15:12:32.118793 5267 leader_election.cc:302] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdc5c807613c4ce5a462f9b6644a8486; no voters:
I20260807 15:12:32.119247 5267 leader_election.cc:288] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 15:12:32.119347 5275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 15:12:32.119546 5275 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [term 3 LEADER]: Becoming Leader. State: Replica: fdc5c807613c4ce5a462f9b6644a8486, State: Running, Role: LEADER
I20260807 15:12:32.119879 5275 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc5c807613c4ce5a462f9b6644a8486" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46183 } }
I20260807 15:12:32.120676 5267 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:12:32.120939 5277 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdc5c807613c4ce5a462f9b6644a8486. Latest consensus state: current_term: 3 leader_uuid: "fdc5c807613c4ce5a462f9b6644a8486" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc5c807613c4ce5a462f9b6644a8486" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46183 } } }
I20260807 15:12:32.121068 5277 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:32.121331 5276 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "fdc5c807613c4ce5a462f9b6644a8486" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdc5c807613c4ce5a462f9b6644a8486" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46183 } } }
I20260807 15:12:32.121445 5276 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdc5c807613c4ce5a462f9b6644a8486 [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:32.121493 5281 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:12:32.127200 5281 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:12:32.128055 5281 catalog_manager.cc:1259] Loaded cluster ID: 970ee3cf882940cca581d7917f8e935c
I20260807 15:12:32.128132 5281 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 15:12:32.129419 5281 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd7753679ad google::LogMessage::Fail() at ??:0
@ 0x7fd77536a88a google::LogMessage::SendToLog() at ??:0
@ 0x7fd7753674de google::LogMessage::Flush() at ??:0
@ 0x7fd775368a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd77fca6fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fd77fca7d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd77fca4579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fd77fce8640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fd7775b93d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd7775b9c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fd7775bb3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fd7775aa512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd778bee6db start_thread at ??:0
@ 0x7fd773e6561f clone at ??:0
W20260807 15:12:32.227147 5271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:32.227600 5271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:32.227674 5271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:32.234361 5271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:32.234493 5271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:32.234591 5271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:12:32.243515 5271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:40159
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=37363
--tserver_master_addrs=127.15.226.126:46183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:32.245129 5271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:32.246878 5271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:32.259625 5334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:32.259795 5271 server_base.cc:1034] running on GCE node
W20260807 15:12:32.259629 5338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:32.259625 5335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:32.260528 5271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:32.261283 5271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:32.262494 5271 hybrid_clock.cc:648] HybridClock initialized: now 1786115552262473 us; error 48 us; skew 500 ppm
Aug 07 15:12:32 dist-test-slave-3kk6 krb5kdc[3522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115552, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:32.265863 5271 init.cc:377] Logged in from keytab as kudu/127.15.226.65@KRBTEST.COM (short username kudu)
I20260807 15:12:32.267108 5271 webserver.cc:460] Webserver started at http://127.15.226.65:37363/ using document root <none> and password file <none>
I20260807 15:12:32.267753 5271 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:32.267864 5271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:32.272295 5271 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:12:32.274853 5352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:32.276332 5271 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:12:32.276505 5271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "53f7598c906449ddad9b0f69ca2278ef"
format_stamp: "Formatted at 2026-08-07 15:12:29 on dist-test-slave-3kk6"
server_key: "f4a6173e00df66104e98268b61984ba7"
server_key_iv: "9d34c4a154e146af42cdc4f5b4023ad7"
server_key_version: "encryptionkey@0"
I20260807 15:12:32.277123 5271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:32.298470 5271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:32.302182 5271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:32.302443 5271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:32.303174 5271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:32.304451 5359 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 15:12:32.310541 5271 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 15:12:32.310698 5271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260807 15:12:32.310802 5271 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 15:12:32.312708 5271 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 15:12:32.312798 5271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260807 15:12:32.313151 5359 tablet_bootstrap.cc:492] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef: Bootstrap starting.
I20260807 15:12:32.324630 5271 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:40159
I20260807 15:12:32.324903 5515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:40159 every 8 connection(s)
I20260807 15:12:32.325824 5271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:12:32.327401 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 5271
I20260807 15:12:32.327913 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:39565
--local_ip_for_outbound_sockets=127.15.226.66
--tserver_master_addrs=127.15.226.126:46183
--webserver_port=34541
--webserver_interface=127.15.226.66
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:12:32.337028 5359 log.cc:824] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:32.344664 5359 tablet_bootstrap.cc:492] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:12:32.345157 5359 tablet_bootstrap.cc:492] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef: Bootstrap complete.
I20260807 15:12:32.346498 5359 ts_tablet_manager.cc:1397] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef: Time spent bootstrapping tablet: real 0.034s user 0.020s sys 0.009s
I20260807 15:12:32.349100 5359 raft_consensus.cc:348] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } }
I20260807 15:12:32.349597 5359 raft_consensus.cc:729] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53f7598c906449ddad9b0f69ca2278ef, State: Initialized, Role: FOLLOWER
I20260807 15:12:32.350136 5359 consensus_queue.cc:260] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } }
I20260807 15:12:32.351238 5359 ts_tablet_manager.cc:1428] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
W20260807 15:12:32.410180 5519 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:12:32.328623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.65:49549)
0807 15:12:32.330099 (+ 1476us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:32.330176 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:32.331562 (+ 1386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:32.409118 (+ 77556us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1197,"thread_start_us":181,"threads_started":1}
W20260807 15:12:32.410854 5516 heartbeater.cc:644] Failed to heartbeat to 127.15.226.126:46183 (0 consecutive failures): Network error: Failed to ping master at 127.15.226.126:46183: Client connection negotiation failed: client connection to 127.15.226.126:46183: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 15:12:32.411799 5519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:32.411442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.65:60767)
0807 15:12:32.411608 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:32.411719 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 15:12:32.412612 5519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:32.412366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.65:39541)
0807 15:12:32.412491 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:32.412550 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 15:12:32.412827 5516 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:12:32.476472 5520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:32.476819 5520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:32.476868 5520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:32.481307 5520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:32.481396 5520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:32.481462 5520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:12:32.487401 5520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:39565
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=34541
--tserver_master_addrs=127.15.226.126:46183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:32.488662 5520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:32.489837 5520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:12:32.501080 5520 server_base.cc:1034] running on GCE node
W20260807 15:12:32.500988 5536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:32.501024 5532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:32.501004 5531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:32.501811 5520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:32.502573 5520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:32.503810 5520 hybrid_clock.cc:648] HybridClock initialized: now 1786115552503788 us; error 40 us; skew 500 ppm
Aug 07 15:12:32 dist-test-slave-3kk6 krb5kdc[3522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115552, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:32.507102 5520 init.cc:377] Logged in from keytab as kudu/127.15.226.66@KRBTEST.COM (short username kudu)
I20260807 15:12:32.508149 5520 webserver.cc:460] Webserver started at http://127.15.226.66:34541/ using document root <none> and password file <none>
I20260807 15:12:32.508762 5520 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:32.508829 5520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:32.513152 5520 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:12:32.515722 5549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:32.517095 5520 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 15:12:32.517262 5520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "5cd7e7e149cf45f5a644b92416699061"
format_stamp: "Formatted at 2026-08-07 15:12:29 on dist-test-slave-3kk6"
server_key: "c768243eab5c25f22a4c992a18fa5735"
server_key_iv: "d81becac286a08a3a5dcf37358603962"
server_key_version: "encryptionkey@0"
I20260807 15:12:32.517899 5520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:32.536168 5520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:32.540221 5520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:32.540488 5520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:32.541260 5520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:32.542266 5520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:32.542320 5520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:32.542397 5520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:32.542430 5520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:32.553231 5520 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:39565
I20260807 15:12:32.553295 5720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:39565 every 8 connection(s)
I20260807 15:12:32.554497 5520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:12:32.559051 5726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:32.557109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.66:56139)
0807 15:12:32.557690 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:32.558191 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":150,"threads_started":1}
I20260807 15:12:32.559598 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 5520
W20260807 15:12:32.559798 5721 heartbeater.cc:644] Failed to heartbeat to 127.15.226.126:46183 (0 consecutive failures): Network error: Failed to ping master at 127.15.226.126:46183: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
I20260807 15:12:32.560158 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:35353
--local_ip_for_outbound_sockets=127.15.226.67
--tserver_master_addrs=127.15.226.126:46183
--webserver_port=40823
--webserver_interface=127.15.226.67
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:38705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:12:32.560659 5726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:32.560341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.66:46353)
0807 15:12:32.560545 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:32.560599 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260807 15:12:32.561372 5726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:32.561167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.66:54805)
0807 15:12:32.561278 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:32.561324 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 15:12:32.561589 5721 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:12:32.685886 5727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:32.686228 5727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:32.686275 5727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:32.690598 5727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:32.690724 5727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:32.690795 5727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:12:32.696805 5727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:38705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:35353
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=40823
--tserver_master_addrs=127.15.226.126:46183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:32.697992 5727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:32.699211 5727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:32.710243 5734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:32.710469 5727 server_base.cc:1034] running on GCE node
W20260807 15:12:32.710497 5735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:32.710208 5737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:32.711115 5727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:32.711916 5727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:32.713115 5727 hybrid_clock.cc:648] HybridClock initialized: now 1786115552713083 us; error 52 us; skew 500 ppm
Aug 07 15:12:32 dist-test-slave-3kk6 krb5kdc[3522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115552, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:32.716610 5727 init.cc:377] Logged in from keytab as kudu/127.15.226.67@KRBTEST.COM (short username kudu)
I20260807 15:12:32.717760 5727 webserver.cc:460] Webserver started at http://127.15.226.67:40823/ using document root <none> and password file <none>
I20260807 15:12:32.718387 5727 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:32.718458 5727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:32.723213 5727 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:12:32.725749 5746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:32.727335 5727 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:12:32.727509 5727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "e5491767f24d4ac5812d23cf91db8d0a"
format_stamp: "Formatted at 2026-08-07 15:12:30 on dist-test-slave-3kk6"
server_key: "2c6a399c214f7bb4cb9bd04690b07931"
server_key_iv: "f10174211d2c301d86b53729415a44bc"
server_key_version: "encryptionkey@0"
I20260807 15:12:32.728212 5727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:32.742504 5727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:32.746466 5727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:32.746784 5727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:32.747526 5727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:32.748524 5727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:32.748615 5727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:32.748713 5727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:32.748767 5727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:32.759207 5727 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:35353
I20260807 15:12:32.759238 5909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:35353 every 8 connection(s)
I20260807 15:12:32.760414 5727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:12:32.765390 5914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:32.763229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.67:55909)
0807 15:12:32.763854 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:32.764002 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":148,"threads_started":1}
W20260807 15:12:32.766013 5911 heartbeater.cc:644] Failed to heartbeat to 127.15.226.126:46183 (0 consecutive failures): Network error: Failed to ping master at 127.15.226.126:46183: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
I20260807 15:12:32.766921 5914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:32.766540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.67:35925)
0807 15:12:32.766787 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:32.766853 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 15:12:32.767614 5914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:32.767403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.67:53239)
0807 15:12:32.767508 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:32.767557 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 15:12:32.767823 5911 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 15:12:32.769951 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 5727
I20260807 15:12:33.414137 5917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:33.413574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.65:58209)
0807 15:12:33.413940 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:33.414015 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":93,"threads_started":1}
I20260807 15:12:33.562878 5918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:33.562317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.66:52667)
0807 15:12:33.562622 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:33.562762 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":87,"threads_started":1}
I20260807 15:12:33.769160 5920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:33.768597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.67:45265)
0807 15:12:33.768960 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:33.769033 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":123,"threads_started":1}
I20260807 15:12:34.172158 5921 raft_consensus.cc:482] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:12:34.172473 5921 raft_consensus.cc:504] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } }
I20260807 15:12:34.173792 5921 leader_election.cc:288] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5491767f24d4ac5812d23cf91db8d0a (127.15.226.67:35353), 5cd7e7e149cf45f5a644b92416699061 (127.15.226.66:39565)
Aug 07 15:12:34 dist-test-slave-3kk6 krb5kdc[3522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115552, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.67@KRBTEST.COM
Aug 07 15:12:34 dist-test-slave-3kk6 krb5kdc[3522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115552, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.66@KRBTEST.COM
I20260807 15:12:34.188712 5929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:34.174372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:39565 (local address 127.15.226.65:41525)
0807 15:12:34.174711 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:34.174724 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:34.174861 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:34.176326 (+ 1465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:34.176329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:34.176817 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:34.177270 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:34.177284 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:34.178609 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:34.178612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:34.179614 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:34.179625 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:34.180352 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:34.180356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:34.180559 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:34.181383 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:34.181399 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:34.183634 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:34.186499 (+ 2865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:34.186501 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:34.186503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:34.186786 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:34.187152 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:34.187156 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:34.187158 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:34.187244 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:34.187918 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:34.187921 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:34.188029 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 15:12:34.188135 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 15:12:34.188498 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":123,"spinlock_wait_cycles":6528,"thread_start_us":58,"threads_started":1}
I20260807 15:12:34.188772 5927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:34.174188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:44327 (local address 127.15.226.67:35353)
0807 15:12:34.174749 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:34.174775 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:34.174816 (+ 41us) server_negotiation.cc:407] Connection header received
0807 15:12:34.175533 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:34.175539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:34.175886 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:34.176136 (+ 250us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:34.177477 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:34.178455 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:34.179750 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:34.180241 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:34.180432 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:34.183499 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:34.183504 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:34.183516 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:34.183547 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:34.185878 (+ 2331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:34.186455 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:34.186464 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:34.186472 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:34.186698 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:34.187076 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:34.187081 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:34.187083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:34.187543 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:34.187700 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:34.188250 (+ 550us) server_negotiation.cc:299] Negotiation successful
0807 15:12:34.188571 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":106,"threads_started":1}
I20260807 15:12:34.188733 5930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:34.174607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:41525 (local address 127.15.226.66:39565)
0807 15:12:34.175077 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:34.175088 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:34.175118 (+ 30us) server_negotiation.cc:407] Connection header received
0807 15:12:34.175740 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:34.175746 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:34.176028 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:34.176353 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:34.177456 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:34.178454 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:34.179745 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:34.180225 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:34.180409 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:34.183870 (+ 3461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:34.183879 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:34.183896 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:34.183943 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:34.186390 (+ 2447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:34.186915 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:34.186928 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:34.186932 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:34.187049 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:34.187364 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:34.187367 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:34.187369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:34.187808 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:34.187947 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:34.188301 (+ 354us) server_negotiation.cc:299] Negotiation successful
0807 15:12:34.188569 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":160,"threads_started":1}
I20260807 15:12:34.188752 5923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:34.174002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:35353 (local address 127.15.226.65:44327)
0807 15:12:34.174275 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:34.174295 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:34.174418 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:34.176257 (+ 1839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:34.176263 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:34.176846 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:34.177269 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:34.177284 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:34.178612 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:34.178622 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:34.179614 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:34.179625 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:34.180401 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:34.180404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:34.180607 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:34.181442 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:34.181454 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:34.183257 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:34.186025 (+ 2768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:34.186030 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:34.186042 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:34.186316 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:34.186816 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:34.186819 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:34.186821 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:34.186969 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:34.187692 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:34.187698 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:34.187932 (+ 234us) client_negotiation.cc:769] Sending connection context
0807 15:12:34.188131 (+ 199us) client_negotiation.cc:240] Negotiation successful
0807 15:12:34.188588 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":234,"spinlock_wait_cycles":1920,"thread_start_us":71,"threads_started":1}
W20260807 15:12:34.191893 5354 leader_election.cc:341] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cd7e7e149cf45f5a644b92416699061 (127.15.226.66:39565): Not found: Tablet not found: 62efbed6bd224fd0bca0fede08358f6f
W20260807 15:12:34.192138 5354 leader_election.cc:341] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e5491767f24d4ac5812d23cf91db8d0a (127.15.226.67:35353): Not found: Tablet not found: 62efbed6bd224fd0bca0fede08358f6f
I20260807 15:12:34.192209 5354 leader_election.cc:302] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53f7598c906449ddad9b0f69ca2278ef; no voters: 5cd7e7e149cf45f5a644b92416699061, e5491767f24d4ac5812d23cf91db8d0a
I20260807 15:12:34.192529 5921 raft_consensus.cc:2726] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:12:34.415516 5929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:34.415140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.65:38385)
0807 15:12:34.415335 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:34.415414 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 15:12:34.564239 5931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:34.563785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.66:59577)
0807 15:12:34.564052 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:34.564113 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":85,"threads_started":1}
I20260807 15:12:34.770579 5932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:34.770099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.67:49485)
0807 15:12:34.770404 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:34.770476 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":88,"threads_started":1}
I20260807 15:12:35.417001 5933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:35.416497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.65:59769)
0807 15:12:35.416812 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:35.416888 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":94,"threads_started":1}
I20260807 15:12:35.565902 5934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:35.565292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.66:40027)
0807 15:12:35.565656 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:35.565748 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":97,"threads_started":1}
I20260807 15:12:35.744860 5936 raft_consensus.cc:482] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:12:35.745083 5936 raft_consensus.cc:504] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } }
I20260807 15:12:35.745673 5936 leader_election.cc:288] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5491767f24d4ac5812d23cf91db8d0a (127.15.226.67:35353), 5cd7e7e149cf45f5a644b92416699061 (127.15.226.66:39565)
W20260807 15:12:35.746835 5354 leader_election.cc:341] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e5491767f24d4ac5812d23cf91db8d0a (127.15.226.67:35353): Not found: Tablet not found: 62efbed6bd224fd0bca0fede08358f6f
W20260807 15:12:35.747017 5354 leader_election.cc:341] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cd7e7e149cf45f5a644b92416699061 (127.15.226.66:39565): Not found: Tablet not found: 62efbed6bd224fd0bca0fede08358f6f
I20260807 15:12:35.747128 5354 leader_election.cc:302] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53f7598c906449ddad9b0f69ca2278ef; no voters: 5cd7e7e149cf45f5a644b92416699061, e5491767f24d4ac5812d23cf91db8d0a
I20260807 15:12:35.747284 5936 raft_consensus.cc:2726] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:12:35.772019 5938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:35.771529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.67:39257)
0807 15:12:35.771843 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:35.771914 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":121,"threads_started":1}
I20260807 15:12:36.418641 5940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:36.418056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.65:59839)
0807 15:12:36.418411 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:36.418500 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":112,"threads_started":1}
I20260807 15:12:36.567245 5943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:36.566798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.66:59823)
0807 15:12:36.567079 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:36.567144 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":88,"threads_started":1}
I20260807 15:12:36.773602 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:36.773067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.67:36181)
0807 15:12:36.773418 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:36.773494 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":128,"threads_started":1}
I20260807 15:12:37.420532 5946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:37.419984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.65:54999)
0807 15:12:37.420351 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:37.420427 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":96,"threads_started":1}
I20260807 15:12:37.568723 5949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:37.568199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.66:36037)
0807 15:12:37.568562 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:37.568619 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":127,"threads_started":1}
I20260807 15:12:37.775168 5950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:37.774611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.67:42247)
0807 15:12:37.774993 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:37.775064 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":191,"threads_started":1}
I20260807 15:12:38.422389 5951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:38.421704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.65:47471)
0807 15:12:38.422090 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:38.422255 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":95,"threads_started":1}
I20260807 15:12:38.570268 5952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:38.569710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.66:40181)
0807 15:12:38.570071 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:38.570145 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":137,"threads_started":1}
I20260807 15:12:38.746796 5954 raft_consensus.cc:482] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:12:38.746960 5954 raft_consensus.cc:504] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } }
I20260807 15:12:38.747467 5954 leader_election.cc:288] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5491767f24d4ac5812d23cf91db8d0a (127.15.226.67:35353), 5cd7e7e149cf45f5a644b92416699061 (127.15.226.66:39565)
W20260807 15:12:38.748384 5354 leader_election.cc:341] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e5491767f24d4ac5812d23cf91db8d0a (127.15.226.67:35353): Not found: Tablet not found: 62efbed6bd224fd0bca0fede08358f6f
W20260807 15:12:38.748553 5354 leader_election.cc:341] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cd7e7e149cf45f5a644b92416699061 (127.15.226.66:39565): Not found: Tablet not found: 62efbed6bd224fd0bca0fede08358f6f
I20260807 15:12:38.748613 5354 leader_election.cc:302] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53f7598c906449ddad9b0f69ca2278ef; no voters: 5cd7e7e149cf45f5a644b92416699061, e5491767f24d4ac5812d23cf91db8d0a
I20260807 15:12:38.748778 5954 raft_consensus.cc:2726] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:12:38.776809 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:38.776289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.67:52269)
0807 15:12:38.776589 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:38.776687 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":128,"threads_started":1}
I20260807 15:12:39.423951 5958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:39.423417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.65:42835)
0807 15:12:39.423738 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:39.423814 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":86,"threads_started":1}
I20260807 15:12:39.572145 5959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:39.571536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.66:47687)
0807 15:12:39.571963 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:39.572039 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":140,"threads_started":1}
I20260807 15:12:39.778405 5964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:39.777735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.67:49479)
0807 15:12:39.778175 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:39.778289 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":97,"threads_started":1}
I20260807 15:12:40.425446 5965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:40.424941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.65:36975)
0807 15:12:40.425263 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:40.425339 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":132,"threads_started":1}
I20260807 15:12:40.573745 5966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:40.573222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.66:36277)
0807 15:12:40.573562 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:40.573638 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":105,"threads_started":1}
I20260807 15:12:40.780105 5969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:40.779499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.67:36697)
0807 15:12:40.779858 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:40.779968 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":125,"threads_started":1}
I20260807 15:12:41.195606 5970 raft_consensus.cc:482] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:12:41.195843 5970 raft_consensus.cc:504] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } }
I20260807 15:12:41.196257 5970 leader_election.cc:288] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5491767f24d4ac5812d23cf91db8d0a (127.15.226.67:35353), 5cd7e7e149cf45f5a644b92416699061 (127.15.226.66:39565)
W20260807 15:12:41.197283 5354 leader_election.cc:341] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cd7e7e149cf45f5a644b92416699061 (127.15.226.66:39565): Not found: Tablet not found: 62efbed6bd224fd0bca0fede08358f6f
W20260807 15:12:41.197458 5354 leader_election.cc:341] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e5491767f24d4ac5812d23cf91db8d0a (127.15.226.67:35353): Not found: Tablet not found: 62efbed6bd224fd0bca0fede08358f6f
I20260807 15:12:41.197525 5354 leader_election.cc:302] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53f7598c906449ddad9b0f69ca2278ef; no voters: 5cd7e7e149cf45f5a644b92416699061, e5491767f24d4ac5812d23cf91db8d0a
I20260807 15:12:41.197741 5970 raft_consensus.cc:2726] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:12:41.426973 5971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:41.426368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.65:52683)
0807 15:12:41.426778 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:41.426864 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":101,"threads_started":1}
I20260807 15:12:41.575524 5973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:41.574948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.66:50261)
0807 15:12:41.575318 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:41.575408 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":95,"threads_started":1}
I20260807 15:12:41.781765 5975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:41.781105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.67:52975)
0807 15:12:41.781502 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:41.781579 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":180,"threads_started":1}
I20260807 15:12:42.428601 5976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:42.428058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.65:34985)
0807 15:12:42.428401 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:42.428492 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":105,"threads_started":1}
I20260807 15:12:42.577147 5979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:42.576605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.66:56297)
0807 15:12:42.576960 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:42.577035 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":118,"threads_started":1}
I20260807 15:12:42.783336 5980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:42.782825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.67:47637)
0807 15:12:42.783138 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:42.783206 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":135,"threads_started":1}
I20260807 15:12:43.430255 5981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:43.429752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.65:46175)
0807 15:12:43.430067 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:43.430139 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":92,"threads_started":1}
I20260807 15:12:43.578439 5982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:43.577992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.66:54133)
0807 15:12:43.578269 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:43.578334 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":87,"threads_started":1}
I20260807 15:12:43.784567 5984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:43.784127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.67:40901)
0807 15:12:43.784399 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:43.784462 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":90,"threads_started":1}
I20260807 15:12:44.431846 5985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:44.431275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.65:55837)
0807 15:12:44.431635 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:44.431737 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":99,"threads_started":1}
I20260807 15:12:44.580108 5986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:44.579525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.66:45867)
0807 15:12:44.579909 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:44.579993 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":102,"threads_started":1}
I20260807 15:12:44.786233 5987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:44.785648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.67:41621)
0807 15:12:44.786023 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:44.786087 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":159,"threads_started":1}
I20260807 15:12:45.433362 5990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:45.432884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.65:38147)
0807 15:12:45.433186 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:45.433258 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":95,"threads_started":1}
I20260807 15:12:45.581993 5991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:45.581276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.66:49885)
0807 15:12:45.581735 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:45.581816 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":107,"threads_started":1}
I20260807 15:12:45.787700 5992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:45.787233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.67:34571)
0807 15:12:45.787525 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:45.787586 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":92,"threads_started":1}
I20260807 15:12:46.296353 5993 raft_consensus.cc:482] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:12:46.296657 5993 raft_consensus.cc:504] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5491767f24d4ac5812d23cf91db8d0a" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 35353 } } peers { permanent_uuid: "5cd7e7e149cf45f5a644b92416699061" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 39565 } } peers { permanent_uuid: "53f7598c906449ddad9b0f69ca2278ef" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 40159 } }
I20260807 15:12:46.297187 5993 leader_election.cc:288] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e5491767f24d4ac5812d23cf91db8d0a (127.15.226.67:35353), 5cd7e7e149cf45f5a644b92416699061 (127.15.226.66:39565)
W20260807 15:12:46.298198 5354 leader_election.cc:341] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e5491767f24d4ac5812d23cf91db8d0a (127.15.226.67:35353): Not found: Tablet not found: 62efbed6bd224fd0bca0fede08358f6f
W20260807 15:12:46.298363 5354 leader_election.cc:341] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5cd7e7e149cf45f5a644b92416699061 (127.15.226.66:39565): Not found: Tablet not found: 62efbed6bd224fd0bca0fede08358f6f
I20260807 15:12:46.298429 5354 leader_election.cc:302] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 53f7598c906449ddad9b0f69ca2278ef; no voters: 5cd7e7e149cf45f5a644b92416699061, e5491767f24d4ac5812d23cf91db8d0a
I20260807 15:12:46.298638 5993 raft_consensus.cc:2726] T 62efbed6bd224fd0bca0fede08358f6f P 53f7598c906449ddad9b0f69ca2278ef [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:12:46.435004 5997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:46.434376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.65:39649)
0807 15:12:46.434815 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:46.434892 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":129,"threads_started":1}
I20260807 15:12:46.583546 5998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:46.583029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.66:33407)
0807 15:12:46.583368 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:46.583442 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":127,"threads_started":1}
I20260807 15:12:46.789327 5999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:46.788800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.67:35337)
0807 15:12:46.789149 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:46.789221 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":145,"threads_started":1}
I20260807 15:12:47.436555 6000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:47.436110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.65:58299)
0807 15:12:47.436389 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:47.436452 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":102,"threads_started":1}
I20260807 15:12:47.585176 6004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:47.584700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.66:35813)
0807 15:12:47.585000 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:47.585072 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":111,"threads_started":1}
I20260807 15:12:47.777161 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 5271
I20260807 15:12:47.790930 6005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:47.790441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46183 (local address 127.15.226.67:46705)
0807 15:12:47.790754 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:47.790824 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:46183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":103,"threads_started":1}
I20260807 15:12:47.793947 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 5520
I20260807 15:12:47.823261 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 5727
2026-08-07T15:12:47Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (20085 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:12:47 dist-test-slave-3kk6 krb5kdc[6008](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:12:47 dist-test-slave-3kk6 krb5kdc[6008](info): set up 2 sockets
Aug 07 15:12:47 dist-test-slave-3kk6 krb5kdc[6008](info): commencing operation
krb5kdc: starting...
W20260807 15:12:49.096148 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:12:49 dist-test-slave-3kk6 krb5kdc[6008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:12:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:12:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:12:49.237486 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:36145
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:44395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:36145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:49.357771 6025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:49.358165 6025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:49.358242 6025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:49.362455 6025 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 15:12:49.362569 6025 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 15:12:49.362596 6025 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 15:12:49.362619 6025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:12:49.362680 6025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:49.362749 6025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:49.362787 6025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:12:49.362823 6025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:12:49.368448 6025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:44395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:36145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:36145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:49.369671 6025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:49.370800 6025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:49.379985 6044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:49.380175 6043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:49.379985 6046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:49.380360 6025 server_base.cc:1034] running on GCE node
I20260807 15:12:49.381003 6025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:49.382194 6025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:49.383366 6025 hybrid_clock.cc:648] HybridClock initialized: now 1786115569383348 us; error 34 us; skew 500 ppm
Aug 07 15:12:49 dist-test-slave-3kk6 krb5kdc[6008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115569, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:49.386404 6025 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:12:49.387434 6025 webserver.cc:460] Webserver started at http://127.15.226.126:36381/ using document root <none> and password file <none>
I20260807 15:12:49.387979 6025 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:49.388042 6025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:49.388214 6025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:49.390336 6025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "dc6bb68a43ef448383e50f723c502b67"
format_stamp: "Formatted at 2026-08-07 15:12:49 on dist-test-slave-3kk6"
server_key: "b80bb9a26f96c1d29ca3878ae9a0e7ca"
server_key_iv: "8f20d99e5b71f1281f1845bbc3ee10b0"
server_key_version: "encryptionkey@0"
I20260807 15:12:49.390972 6025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "dc6bb68a43ef448383e50f723c502b67"
format_stamp: "Formatted at 2026-08-07 15:12:49 on dist-test-slave-3kk6"
server_key: "b80bb9a26f96c1d29ca3878ae9a0e7ca"
server_key_iv: "8f20d99e5b71f1281f1845bbc3ee10b0"
server_key_version: "encryptionkey@0"
I20260807 15:12:49.394804 6025 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:12:49.397468 6057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:49.398851 6025 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:12:49.399010 6025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "dc6bb68a43ef448383e50f723c502b67"
format_stamp: "Formatted at 2026-08-07 15:12:49 on dist-test-slave-3kk6"
server_key: "b80bb9a26f96c1d29ca3878ae9a0e7ca"
server_key_iv: "8f20d99e5b71f1281f1845bbc3ee10b0"
server_key_version: "encryptionkey@0"
I20260807 15:12:49.399138 6025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:49.426918 6025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:49.430171 6025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:49.430418 6025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:49.438164 6025 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:36145
I20260807 15:12:49.438509 6129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:36145 every 8 connection(s)
I20260807 15:12:49.439278 6025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:12:49.442246 6130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:49.445853 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 6025
I20260807 15:12:49.446005 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:12:49.446326 16265 external_mini_cluster.cc:1428] Setting key 9221938845bcebf8b689ada0c38acde0
I20260807 15:12:49.448967 6130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67: Bootstrap starting.
I20260807 15:12:49.452235 6130 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:49.453240 6130 log.cc:824] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:49.455955 6130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67: No bootstrap required, opened a new log
Aug 07 15:12:49 dist-test-slave-3kk6 krb5kdc[6008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:12:49.459452 6130 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc6bb68a43ef448383e50f723c502b67" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36145 } }
I20260807 15:12:49.459771 6130 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:49.459862 6130 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc6bb68a43ef448383e50f723c502b67, State: Initialized, Role: FOLLOWER
I20260807 15:12:49.460400 6130 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc6bb68a43ef448383e50f723c502b67" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36145 } }
I20260807 15:12:49.460568 6130 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:12:49.460661 6130 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:12:49.460789 6130 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:49.461833 6130 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc6bb68a43ef448383e50f723c502b67" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36145 } }
I20260807 15:12:49.462231 6130 leader_election.cc:302] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc6bb68a43ef448383e50f723c502b67; no voters:
I20260807 15:12:49.462711 6130 leader_election.cc:288] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:12:49.462841 6137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:12:49.463055 6137 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 1 LEADER]: Becoming Leader. State: Replica: dc6bb68a43ef448383e50f723c502b67, State: Running, Role: LEADER
I20260807 15:12:49.463480 6137 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc6bb68a43ef448383e50f723c502b67" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36145 } }
I20260807 15:12:49.463851 6134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:49.447929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60364 (local address 127.15.226.126:36145)
0807 15:12:49.448380 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:49.448396 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:49.448444 (+ 48us) server_negotiation.cc:407] Connection header received
0807 15:12:49.449338 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:49.449364 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:49.449920 (+ 556us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:49.450393 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:49.451444 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:49.452594 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:49.453838 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:49.454513 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:49.454640 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:49.457646 (+ 3006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:49.457652 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:49.457666 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:49.457695 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:49.460321 (+ 2626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:49.460916 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:49.460921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:49.460926 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:49.461050 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:49.461365 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:49.461369 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:49.461371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:49.461805 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:49.462022 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:49.462357 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 15:12:49.462757 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":151,"threads_started":1}
I20260807 15:12:49.464161 6130 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:12:49.464800 6137 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc6bb68a43ef448383e50f723c502b67. Latest consensus state: current_term: 1 leader_uuid: "dc6bb68a43ef448383e50f723c502b67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc6bb68a43ef448383e50f723c502b67" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36145 } } }
I20260807 15:12:49.465041 6137 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:49.465248 6138 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc6bb68a43ef448383e50f723c502b67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc6bb68a43ef448383e50f723c502b67" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36145 } } }
I20260807 15:12:49.465355 6138 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [sys.catalog]: This master's current role is: LEADER
I20260807 15:12:49.466193 6142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:12:49.469679 6142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:12:49.475823 6142 catalog_manager.cc:1347] Generated new cluster ID: 6eaf17e2d86d4117b916114ac1d0216c
I20260807 15:12:49.475924 6142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:12:49.486470 6142 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:12:49.487920 6142 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:12:49.499840 6142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67: Generated new TSK 0
I20260807 15:12:49.500844 6142 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.226.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
I20260807 15:12:49.538903 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:36145
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:44395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:49.661698 6175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:49.662053 6175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:49.662101 6175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:49.666711 6175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:49.666792 6175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:49.666914 6175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:12:49.672971 6175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:44395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:36145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:49.674261 6175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:49.675418 6175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:49.686244 6184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:49.686235 6183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:49.686247 6188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:49.686549 6175 server_base.cc:1034] running on GCE node
I20260807 15:12:49.687028 6175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:49.687775 6175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:49.688977 6175 hybrid_clock.cc:648] HybridClock initialized: now 1786115569688962 us; error 29 us; skew 500 ppm
Aug 07 15:12:49 dist-test-slave-3kk6 krb5kdc[6008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115569, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:49.692301 6175 init.cc:377] Logged in from keytab as kudu/127.15.226.65@KRBTEST.COM (short username kudu)
I20260807 15:12:49.693359 6175 webserver.cc:460] Webserver started at http://127.15.226.65:35051/ using document root <none> and password file <none>
I20260807 15:12:49.694036 6175 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:49.694141 6175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:49.694403 6175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:49.696720 6175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50"
format_stamp: "Formatted at 2026-08-07 15:12:49 on dist-test-slave-3kk6"
server_key: "b9dbd8a1676b948e52ec2ef492ddb05b"
server_key_iv: "c7b10e4a445bb65768d427d4393b99b9"
server_key_version: "encryptionkey@0"
I20260807 15:12:49.697379 6175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50"
format_stamp: "Formatted at 2026-08-07 15:12:49 on dist-test-slave-3kk6"
server_key: "b9dbd8a1676b948e52ec2ef492ddb05b"
server_key_iv: "c7b10e4a445bb65768d427d4393b99b9"
server_key_version: "encryptionkey@0"
I20260807 15:12:49.701768 6175 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:12:49.704794 6200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:49.706099 6175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:12:49.706261 6175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50"
format_stamp: "Formatted at 2026-08-07 15:12:49 on dist-test-slave-3kk6"
server_key: "b9dbd8a1676b948e52ec2ef492ddb05b"
server_key_iv: "c7b10e4a445bb65768d427d4393b99b9"
server_key_version: "encryptionkey@0"
I20260807 15:12:49.706405 6175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:49.724756 6175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:49.728551 6175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:49.728801 6175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:49.729483 6175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:49.730408 6175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:49.730484 6175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:49.730562 6175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:49.730607 6175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:49.741209 6175 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:36265
I20260807 15:12:49.741237 6356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:36265 every 8 connection(s)
I20260807 15:12:49.742189 6175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:12:49.747888 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 6175
I20260807 15:12:49.748044 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:12:49.748413 16265 external_mini_cluster.cc:1428] Setting key 93f1f28b4d41bea478c604deb8f79a71
Aug 07 15:12:49 dist-test-slave-3kk6 krb5kdc[6008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115569, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:12:49.757942 6134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:49.744394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:35475 (local address 127.15.226.126:36145)
0807 15:12:49.744551 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:49.744555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:49.745455 (+ 900us) server_negotiation.cc:407] Connection header received
0807 15:12:49.746928 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:49.746931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:49.746981 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:49.747061 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:49.748741 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:49.749327 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:49.750715 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:49.750964 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:49.751084 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:49.754027 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:49.754030 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:49.754033 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:49.754056 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:49.755759 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:49.756313 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:49.756317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:49.756318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:49.756406 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:49.756773 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:49.756777 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:49.756779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:49.756979 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:49.757091 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:49.757621 (+ 530us) server_negotiation.cc:299] Negotiation successful
0807 15:12:49.757766 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 15:12:49.758949 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:49.744794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.65:35475)
0807 15:12:49.745284 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:49.745331 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:49.746627 (+ 1296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:49.747240 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:49.747247 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:49.747789 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:49.748538 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:49.748563 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:49.749480 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:49.749484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:49.750554 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:49.750565 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:49.751134 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:49.751139 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:49.751316 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:49.752020 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:49.752038 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:49.753833 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:49.755894 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:49.755900 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:49.755918 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:49.756199 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:49.756504 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:49.756507 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:49.756509 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:49.756660 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:49.757091 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:49.757099 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:49.757374 (+ 275us) client_negotiation.cc:769] Sending connection context
0807 15:12:49.757577 (+ 203us) client_negotiation.cc:240] Negotiation successful
0807 15:12:49.757954 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":121,"threads_started":1}
WARNING: no policy specified for kudu/127.15.226.66@KRBTEST.COM; defaulting to no policy
I20260807 15:12:49.760515 6359 heartbeater.cc:347] Connected to a master server at 127.15.226.126:36145
I20260807 15:12:49.760849 6359 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:49.761400 6359 heartbeater.cc:511] Master 127.15.226.126:36145 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.66@KRBTEST.COM" created.
I20260807 15:12:49.763332 6083 ts_manager.cc:194] Registered new tserver with Master: 3f1e4a721b8f4c6e8ff8325b0e48db50 (127.15.226.65:36265)
I20260807 15:12:49.764587 6083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.65@KRBTEST.COM'} at 127.15.226.65:35475
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
I20260807 15:12:49.783007 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:36145
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:44395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:49.905055 6367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:49.905380 6367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:49.905426 6367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:49.909633 6367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:49.909777 6367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:49.909888 6367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:12:49.915793 6367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:44395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:36145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:49.916915 6367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:49.917939 6367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:49.928085 6374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:49.928086 6376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:49.928418 6367 server_base.cc:1034] running on GCE node
W20260807 15:12:49.928108 6378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:49.928894 6367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:49.929659 6367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:49.930860 6367 hybrid_clock.cc:648] HybridClock initialized: now 1786115569930839 us; error 35 us; skew 500 ppm
Aug 07 15:12:49 dist-test-slave-3kk6 krb5kdc[6008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115569, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:49.933866 6367 init.cc:377] Logged in from keytab as kudu/127.15.226.66@KRBTEST.COM (short username kudu)
I20260807 15:12:49.934933 6367 webserver.cc:460] Webserver started at http://127.15.226.66:46525/ using document root <none> and password file <none>
I20260807 15:12:49.935556 6367 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:49.935657 6367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:49.935905 6367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:49.938153 6367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "caee2f5650aa4f1e8ae25fc770229752"
format_stamp: "Formatted at 2026-08-07 15:12:49 on dist-test-slave-3kk6"
server_key: "45b4eb4fcd13fda47a072f9ff244adf8"
server_key_iv: "8b162e95b7d5f2e0ddcb7adedac46454"
server_key_version: "encryptionkey@0"
I20260807 15:12:49.938880 6367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "caee2f5650aa4f1e8ae25fc770229752"
format_stamp: "Formatted at 2026-08-07 15:12:49 on dist-test-slave-3kk6"
server_key: "45b4eb4fcd13fda47a072f9ff244adf8"
server_key_iv: "8b162e95b7d5f2e0ddcb7adedac46454"
server_key_version: "encryptionkey@0"
I20260807 15:12:49.943056 6367 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:12:49.945878 6387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:49.947422 6367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:12:49.947592 6367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "caee2f5650aa4f1e8ae25fc770229752"
format_stamp: "Formatted at 2026-08-07 15:12:49 on dist-test-slave-3kk6"
server_key: "45b4eb4fcd13fda47a072f9ff244adf8"
server_key_iv: "8b162e95b7d5f2e0ddcb7adedac46454"
server_key_version: "encryptionkey@0"
I20260807 15:12:49.947739 6367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:49.978096 6367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:49.981686 6367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:49.981938 6367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:49.982618 6367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:49.983608 6367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:49.983685 6367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:49.983762 6367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:49.983806 6367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:49.993577 6367 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:43769
I20260807 15:12:49.993623 6549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:43769 every 8 connection(s)
I20260807 15:12:49.994607 6367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:12:50.002424 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 6367
I20260807 15:12:50.002573 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:12:50.002943 16265 external_mini_cluster.cc:1428] Setting key 6f9ec165e739d78e502d05b5d86e87d2
Aug 07 15:12:49 dist-test-slave-3kk6 krb5kdc[6008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115569, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:12:50.012744 6134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:49.996779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:57101 (local address 127.15.226.126:36145)
0807 15:12:49.996950 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:49.996954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:49.997899 (+ 945us) server_negotiation.cc:407] Connection header received
0807 15:12:49.999479 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:49.999482 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:49.999535 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:49.999603 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:50.001072 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.001625 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.003445 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.003812 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.003906 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:50.007389 (+ 3483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:50.007396 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:50.007401 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:50.007443 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:50.009567 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:50.010446 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:50.010453 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:50.010458 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:50.010584 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:50.011187 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:50.011192 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:50.011194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:50.011499 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:50.011619 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:50.012288 (+ 669us) server_negotiation.cc:299] Negotiation successful
0807 15:12:50.012476 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 15:12:50.013478 6555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:49.997101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.66:57101)
0807 15:12:49.997652 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:49.997704 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:49.999252 (+ 1548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:49.999809 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:49.999816 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:50.000303 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:50.000886 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:50.000903 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.001796 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.001800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:50.003264 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:50.003297 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.003985 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.003990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:50.004196 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:50.004966 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:50.004983 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:50.007099 (+ 2116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:50.009747 (+ 2648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:50.009842 (+ 95us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:50.009866 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:50.010293 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:50.010739 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:50.010742 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:50.010812 (+ 70us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:50.011045 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:50.011619 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:50.011625 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:50.011972 (+ 347us) client_negotiation.cc:769] Sending connection context
0807 15:12:50.012151 (+ 179us) client_negotiation.cc:240] Negotiation successful
0807 15:12:50.012476 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":135,"threads_started":1}
I20260807 15:12:50.015054 6551 heartbeater.cc:347] Connected to a master server at 127.15.226.126:36145
I20260807 15:12:50.015384 6551 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.15.226.67@KRBTEST.COM; defaulting to no policy
I20260807 15:12:50.016072 6551 heartbeater.cc:511] Master 127.15.226.126:36145 requested a full tablet report, sending...
I20260807 15:12:50.017383 6083 ts_manager.cc:194] Registered new tserver with Master: caee2f5650aa4f1e8ae25fc770229752 (127.15.226.66:43769)
I20260807 15:12:50.018178 6083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.66@KRBTEST.COM'} at 127.15.226.66:57101
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
I20260807 15:12:50.038808 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:36145
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:44395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:12:50.165366 6561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:12:50.165692 6561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:12:50.165740 6561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:12:50.170135 6561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:12:50.170229 6561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:12:50.170293 6561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:12:50.175987 6561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:44395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:36145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:12:50.177130 6561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:12:50.178144 6561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:12:50.188592 6568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:12:50.188601 6570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:50.188678 6561 server_base.cc:1034] running on GCE node
W20260807 15:12:50.188630 6567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:12:50.189344 6561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:12:50.190105 6561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:12:50.191301 6561 hybrid_clock.cc:648] HybridClock initialized: now 1786115570191287 us; error 34 us; skew 500 ppm
Aug 07 15:12:50 dist-test-slave-3kk6 krb5kdc[6008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115570, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:12:50.194481 6561 init.cc:377] Logged in from keytab as kudu/127.15.226.67@KRBTEST.COM (short username kudu)
I20260807 15:12:50.195575 6561 webserver.cc:460] Webserver started at http://127.15.226.67:43265/ using document root <none> and password file <none>
I20260807 15:12:50.196188 6561 fs_manager.cc:359] Metadata directory not provided
I20260807 15:12:50.196286 6561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:12:50.196568 6561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:12:50.198863 6561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "489972c8b9ef4a339aa8c09de646d108"
format_stamp: "Formatted at 2026-08-07 15:12:50 on dist-test-slave-3kk6"
server_key: "b0b1cd8b3d93b9f51071529b3be3a8dc"
server_key_iv: "7c90822e7528e0766090101f0f863fad"
server_key_version: "encryptionkey@0"
I20260807 15:12:50.199505 6561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "489972c8b9ef4a339aa8c09de646d108"
format_stamp: "Formatted at 2026-08-07 15:12:50 on dist-test-slave-3kk6"
server_key: "b0b1cd8b3d93b9f51071529b3be3a8dc"
server_key_iv: "7c90822e7528e0766090101f0f863fad"
server_key_version: "encryptionkey@0"
I20260807 15:12:50.203689 6561 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:12:50.206452 6578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:50.207939 6561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:12:50.208092 6561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "489972c8b9ef4a339aa8c09de646d108"
format_stamp: "Formatted at 2026-08-07 15:12:50 on dist-test-slave-3kk6"
server_key: "b0b1cd8b3d93b9f51071529b3be3a8dc"
server_key_iv: "7c90822e7528e0766090101f0f863fad"
server_key_version: "encryptionkey@0"
I20260807 15:12:50.208225 6561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:12:50.233373 6561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:12:50.237164 6561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:12:50.237416 6561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:12:50.238116 6561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:12:50.239142 6561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:12:50.239220 6561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:50.239303 6561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:12:50.239359 6561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:12:50.249683 6561 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:46649
I20260807 15:12:50.249711 6749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:46649 every 8 connection(s)
I20260807 15:12:50.250789 6561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:12:50.257907 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 6561
I20260807 15:12:50.258037 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:12:50.258463 16265 external_mini_cluster.cc:1428] Setting key 9a9be7a117b993df3a5b78b111c982f6
Aug 07 15:12:50 dist-test-slave-3kk6 krb5kdc[6008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115570, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:12:50.267494 6134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:50.253152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:44437 (local address 127.15.226.126:36145)
0807 15:12:50.253328 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:50.253333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:50.254337 (+ 1004us) server_negotiation.cc:407] Connection header received
0807 15:12:50.255907 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:50.255910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:50.255962 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:50.256046 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:50.257454 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.258099 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.259985 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.260344 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.260460 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:50.263538 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:50.263541 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:50.263543 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:50.263573 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:50.265255 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:50.265822 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:50.265825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:50.265826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:50.265906 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:50.266285 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:50.266287 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:50.266288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:50.266546 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:50.266626 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:50.267236 (+ 610us) server_negotiation.cc:299] Negotiation successful
0807 15:12:50.267365 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:12:50.268428 6753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:50.253605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.67:44437)
0807 15:12:50.254144 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:50.254196 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:50.255664 (+ 1468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:50.256240 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:50.256247 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:50.256674 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:12:50.257282 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:50.257298 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.258247 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.258252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:50.259793 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:50.259808 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.260516 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.260519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:50.260691 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:50.261482 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:12:50.261501 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:12:50.263353 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:12:50.265377 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:50.265384 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:50.265398 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:50.265702 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:50.266017 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:12:50.266020 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:12:50.266021 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:12:50.266183 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:12:50.266645 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:12:50.266683 (+ 38us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:12:50.266964 (+ 281us) client_negotiation.cc:769] Sending connection context
0807 15:12:50.267132 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 15:12:50.267449 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":138,"threads_started":1}
I20260807 15:12:50.269608 6751 heartbeater.cc:347] Connected to a master server at 127.15.226.126:36145
I20260807 15:12:50.269868 6751 heartbeater.cc:464] Registering TS with master...
I20260807 15:12:50.270352 6751 heartbeater.cc:511] Master 127.15.226.126:36145 requested a full tablet report, sending...
I20260807 15:12:50.271473 6083 ts_manager.cc:194] Registered new tserver with Master: 489972c8b9ef4a339aa8c09de646d108 (127.15.226.67:46649)
I20260807 15:12:50.272157 6083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.67@KRBTEST.COM'} at 127.15.226.67:44437
I20260807 15:12:50.273298 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:12:50.283864 6134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:50.275523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60380 (local address 127.15.226.126:36145)
0807 15:12:50.275698 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:50.275705 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:50.275936 (+ 231us) server_negotiation.cc:407] Connection header received
0807 15:12:50.276105 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:50.276108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:50.276158 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:50.276264 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:12:50.277074 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.277652 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.278598 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.278920 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.279044 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:50.280598 (+ 1554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:12:50.280602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:12:50.280605 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:12:50.280641 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:12:50.282010 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:50.282512 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:50.282515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:50.282517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:50.282594 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:12:50.282923 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:12:50.282926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:12:50.282928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:12:50.283162 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:12:50.283308 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:50.283560 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 15:12:50.283681 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 15:12:50.287055 6083 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60380:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:12:50.290300 6083 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:12:50.305938 6765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:50.299763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:36265 (local address 127.0.0.1:51746)
0807 15:12:50.300723 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:50.300739 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:50.300895 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:50.301533 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:50.301539 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:50.301564 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:12:50.301745 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:50.301755 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.303075 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.303080 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:50.304396 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:50.304406 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.305344 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.305355 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:50.305464 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:50.305507 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 15:12:50.305622 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 15:12:50.305767 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"thread_start_us":146,"threads_started":1}
I20260807 15:12:50.305958 6767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:50.299979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51746 (local address 127.15.226.65:36265)
0807 15:12:50.300577 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:50.300584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:50.300820 (+ 236us) server_negotiation.cc:407] Connection header received
0807 15:12:50.301111 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:50.301119 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:50.301366 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:50.301529 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:12:50.301908 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.302961 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.304524 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.305191 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.305309 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:50.305466 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:50.305674 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 15:12:50.305784 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":92,"threads_started":1}
I20260807 15:12:50.306550 6771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:50.300437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:46649 (local address 127.0.0.1:39200)
0807 15:12:50.301095 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:50.301110 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:50.301203 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:50.302015 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:50.302018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:50.302036 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:12:50.302112 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:50.302117 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.303660 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.303662 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:50.304928 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:50.304939 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.306186 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.306189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:50.306254 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:50.306274 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 15:12:50.306333 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 15:12:50.306385 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"mutex_wait_us":39,"thread_start_us":96,"threads_started":1}
I20260807 15:12:50.306823 6766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:50.299612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:43769 (local address 127.0.0.1:50144)
0807 15:12:50.300170 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:50.300236 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:50.300377 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:50.301791 (+ 1414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:50.301794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:50.301811 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:12:50.301931 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:50.301937 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.303570 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.303576 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:50.304928 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:50.304939 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.306269 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.306273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:50.306365 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:50.306397 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 15:12:50.306472 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 15:12:50.306536 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":85,"thread_start_us":108,"threads_started":1}
I20260807 15:12:50.306948 6772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:50.300673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50144 (local address 127.15.226.66:43769)
0807 15:12:50.301233 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:50.301247 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:50.301269 (+ 22us) server_negotiation.cc:407] Connection header received
0807 15:12:50.301416 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:50.301421 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:50.301627 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:50.301790 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:12:50.302076 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.303382 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.305082 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.305918 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.306031 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:50.306193 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:50.306657 (+ 464us) server_negotiation.cc:299] Negotiation successful
0807 15:12:50.306793 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":130,"threads_started":1}
I20260807 15:12:50.307502 6774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:50.300629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39200 (local address 127.15.226.67:46649)
0807 15:12:50.301505 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:50.301518 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:50.301539 (+ 21us) server_negotiation.cc:407] Connection header received
0807 15:12:50.301614 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:50.301619 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:50.301808 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:50.301954 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:12:50.302235 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.303525 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.305074 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.305908 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.306023 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:50.307192 (+ 1169us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:50.307234 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 15:12:50.307305 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741,"thread_start_us":177,"threads_started":1}
I20260807 15:12:50.309965 6650 tablet_service.cc:1467] Processing CreateTablet for tablet 19580d44f1a44a97a2818dee8deda7eb (DEFAULT_TABLE table=test-table [id=e233f0e91c164988ae6db95711aca15c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:12:50.309965 6452 tablet_service.cc:1467] Processing CreateTablet for tablet 19580d44f1a44a97a2818dee8deda7eb (DEFAULT_TABLE table=test-table [id=e233f0e91c164988ae6db95711aca15c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:12:50.309962 6267 tablet_service.cc:1467] Processing CreateTablet for tablet 19580d44f1a44a97a2818dee8deda7eb (DEFAULT_TABLE table=test-table [id=e233f0e91c164988ae6db95711aca15c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:12:50.311093 6267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19580d44f1a44a97a2818dee8deda7eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:50.311100 6650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19580d44f1a44a97a2818dee8deda7eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:50.311093 6452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19580d44f1a44a97a2818dee8deda7eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:12:50.319192 6779 tablet_bootstrap.cc:492] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752: Bootstrap starting.
I20260807 15:12:50.319605 6778 tablet_bootstrap.cc:492] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50: Bootstrap starting.
I20260807 15:12:50.319936 6775 tablet_bootstrap.cc:492] T 19580d44f1a44a97a2818dee8deda7eb P 489972c8b9ef4a339aa8c09de646d108: Bootstrap starting.
I20260807 15:12:50.321774 6778 tablet_bootstrap.cc:654] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:50.322171 6775 tablet_bootstrap.cc:654] T 19580d44f1a44a97a2818dee8deda7eb P 489972c8b9ef4a339aa8c09de646d108: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:50.322247 6779 tablet_bootstrap.cc:654] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752: Neither blocks nor log segments found. Creating new log.
I20260807 15:12:50.322875 6778 log.cc:824] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:50.323009 6775 log.cc:824] T 19580d44f1a44a97a2818dee8deda7eb P 489972c8b9ef4a339aa8c09de646d108: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:50.323092 6779 log.cc:824] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752: Log is configured to *not* fsync() on all Append() calls
I20260807 15:12:50.325325 6779 tablet_bootstrap.cc:492] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752: No bootstrap required, opened a new log
I20260807 15:12:50.325366 6775 tablet_bootstrap.cc:492] T 19580d44f1a44a97a2818dee8deda7eb P 489972c8b9ef4a339aa8c09de646d108: No bootstrap required, opened a new log
I20260807 15:12:50.325556 6779 ts_tablet_manager.cc:1397] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 15:12:50.325562 6775 ts_tablet_manager.cc:1397] T 19580d44f1a44a97a2818dee8deda7eb P 489972c8b9ef4a339aa8c09de646d108: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 15:12:50.325743 6778 tablet_bootstrap.cc:492] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50: No bootstrap required, opened a new log
I20260807 15:12:50.325932 6778 ts_tablet_manager.cc:1397] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 15:12:50.328817 6779 raft_consensus.cc:348] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } } peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } }
I20260807 15:12:50.329110 6779 raft_consensus.cc:374] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:50.329191 6779 raft_consensus.cc:729] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caee2f5650aa4f1e8ae25fc770229752, State: Initialized, Role: FOLLOWER
I20260807 15:12:50.329728 6779 consensus_queue.cc:260] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } } peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } }
I20260807 15:12:50.329885 6775 raft_consensus.cc:348] T 19580d44f1a44a97a2818dee8deda7eb P 489972c8b9ef4a339aa8c09de646d108 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } } peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } }
I20260807 15:12:50.330216 6775 raft_consensus.cc:374] T 19580d44f1a44a97a2818dee8deda7eb P 489972c8b9ef4a339aa8c09de646d108 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:50.330346 6775 raft_consensus.cc:729] T 19580d44f1a44a97a2818dee8deda7eb P 489972c8b9ef4a339aa8c09de646d108 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 489972c8b9ef4a339aa8c09de646d108, State: Initialized, Role: FOLLOWER
I20260807 15:12:50.330636 6551 heartbeater.cc:503] Master 127.15.226.126:36145 was elected leader, sending a full tablet report...
I20260807 15:12:50.330592 6778 raft_consensus.cc:348] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } } peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } }
I20260807 15:12:50.330868 6778 raft_consensus.cc:374] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:12:50.330919 6778 raft_consensus.cc:729] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1e4a721b8f4c6e8ff8325b0e48db50, State: Initialized, Role: FOLLOWER
I20260807 15:12:50.330924 6779 ts_tablet_manager.cc:1428] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 15:12:50.330930 6775 consensus_queue.cc:260] T 19580d44f1a44a97a2818dee8deda7eb P 489972c8b9ef4a339aa8c09de646d108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } } peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } }
I20260807 15:12:50.331430 6778 consensus_queue.cc:260] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } } peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } }
I20260807 15:12:50.332038 6775 ts_tablet_manager.cc:1428] T 19580d44f1a44a97a2818dee8deda7eb P 489972c8b9ef4a339aa8c09de646d108: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260807 15:12:50.332454 6751 heartbeater.cc:503] Master 127.15.226.126:36145 was elected leader, sending a full tablet report...
I20260807 15:12:50.332527 6778 ts_tablet_manager.cc:1428] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 15:12:50.332763 6359 heartbeater.cc:503] Master 127.15.226.126:36145 was elected leader, sending a full tablet report...
I20260807 15:12:50.433334 6789 raft_consensus.cc:482] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:12:50.433640 6789 raft_consensus.cc:504] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } } peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } }
I20260807 15:12:50.435043 6789 leader_election.cc:288] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f1e4a721b8f4c6e8ff8325b0e48db50 (127.15.226.65:36265), 489972c8b9ef4a339aa8c09de646d108 (127.15.226.67:46649)
I20260807 15:12:50.439435 6774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:50.435609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:50715 (local address 127.15.226.67:46649)
0807 15:12:50.435753 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:50.435757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:50.435961 (+ 204us) server_negotiation.cc:407] Connection header received
0807 15:12:50.436076 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:50.436080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:50.436130 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:50.436221 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:12:50.436525 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.437035 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.438265 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.438758 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.438896 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:50.438956 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:50.439186 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 15:12:50.439286 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:12:50.439563 6555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:50.435359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:36265 (local address 127.15.226.66:47893)
0807 15:12:50.435527 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:50.435547 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:50.435659 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:50.435993 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:50.435996 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:50.436036 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:12:50.436200 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:50.436212 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.437077 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.437082 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:50.438138 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:50.438150 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.439086 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.439092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:50.439247 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:50.439269 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 15:12:50.439354 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 15:12:50.439429 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":83}
I20260807 15:12:50.439586 6767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:50.435471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:47893 (local address 127.15.226.65:36265)
0807 15:12:50.435621 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:50.435626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:50.435644 (+ 18us) server_negotiation.cc:407] Connection header received
0807 15:12:50.435786 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:50.435790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:50.435850 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:50.435948 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:12:50.436343 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.436947 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.438293 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.438952 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.439083 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:50.439134 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:50.439384 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 15:12:50.439470 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 15:12:50.439949 6793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:50.435392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:46649 (local address 127.15.226.66:50715)
0807 15:12:50.435857 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:50.435872 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:12:50.435973 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:12:50.436241 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:12:50.436243 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:12:50.436277 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:12:50.436410 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:50.436416 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.437149 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.437151 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:50.438124 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:12:50.438135 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.438895 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.438899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:12:50.439004 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:50.439036 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 15:12:50.439702 (+ 666us) client_negotiation.cc:240] Negotiation successful
0807 15:12:50.439795 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":21,"thread_start_us":156,"threads_started":1}
I20260807 15:12:50.440541 6295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19580d44f1a44a97a2818dee8deda7eb" candidate_uuid: "caee2f5650aa4f1e8ae25fc770229752" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" is_pre_election: true
I20260807 15:12:50.440630 6681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19580d44f1a44a97a2818dee8deda7eb" candidate_uuid: "caee2f5650aa4f1e8ae25fc770229752" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "489972c8b9ef4a339aa8c09de646d108" is_pre_election: true
I20260807 15:12:50.440891 6295 raft_consensus.cc:2454] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate caee2f5650aa4f1e8ae25fc770229752 in term 0.
I20260807 15:12:50.440958 6681 raft_consensus.cc:2454] T 19580d44f1a44a97a2818dee8deda7eb P 489972c8b9ef4a339aa8c09de646d108 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate caee2f5650aa4f1e8ae25fc770229752 in term 0.
I20260807 15:12:50.441452 6395 leader_election.cc:302] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3f1e4a721b8f4c6e8ff8325b0e48db50, 489972c8b9ef4a339aa8c09de646d108, caee2f5650aa4f1e8ae25fc770229752; no voters:
I20260807 15:12:50.441774 6789 raft_consensus.cc:2781] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:12:50.441869 6789 raft_consensus.cc:482] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:12:50.441956 6789 raft_consensus.cc:3037] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:50.443212 6789 raft_consensus.cc:504] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } } peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } }
I20260807 15:12:50.443639 6789 leader_election.cc:288] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752 [CANDIDATE]: Term 1 election: Requested vote from peers 3f1e4a721b8f4c6e8ff8325b0e48db50 (127.15.226.65:36265), 489972c8b9ef4a339aa8c09de646d108 (127.15.226.67:46649)
I20260807 15:12:50.444010 6295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19580d44f1a44a97a2818dee8deda7eb" candidate_uuid: "caee2f5650aa4f1e8ae25fc770229752" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50"
I20260807 15:12:50.444077 6681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19580d44f1a44a97a2818dee8deda7eb" candidate_uuid: "caee2f5650aa4f1e8ae25fc770229752" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "489972c8b9ef4a339aa8c09de646d108"
I20260807 15:12:50.444187 6681 raft_consensus.cc:3037] T 19580d44f1a44a97a2818dee8deda7eb P 489972c8b9ef4a339aa8c09de646d108 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:50.444180 6295 raft_consensus.cc:3037] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:12:50.445561 6295 raft_consensus.cc:2454] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caee2f5650aa4f1e8ae25fc770229752 in term 1.
I20260807 15:12:50.445561 6681 raft_consensus.cc:2454] T 19580d44f1a44a97a2818dee8deda7eb P 489972c8b9ef4a339aa8c09de646d108 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate caee2f5650aa4f1e8ae25fc770229752 in term 1.
I20260807 15:12:50.445923 6395 leader_election.cc:302] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f1e4a721b8f4c6e8ff8325b0e48db50, caee2f5650aa4f1e8ae25fc770229752; no voters:
I20260807 15:12:50.446125 6789 raft_consensus.cc:2781] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:12:50.446332 6789 raft_consensus.cc:686] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752 [term 1 LEADER]: Becoming Leader. State: Replica: caee2f5650aa4f1e8ae25fc770229752, State: Running, Role: LEADER
I20260807 15:12:50.446739 6789 consensus_queue.cc:237] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } } peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } }
I20260807 15:12:50.450538 6083 catalog_manager.cc:5495] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752 reported cstate change: term changed from 0 to 1, leader changed from <none> to caee2f5650aa4f1e8ae25fc770229752 (127.15.226.66). New cstate: current_term: 1 leader_uuid: "caee2f5650aa4f1e8ae25fc770229752" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } health_report { overall_health: UNKNOWN } } }
I20260807 15:12:50.463697 6772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:50.459885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50156 (local address 127.15.226.66:43769)
0807 15:12:50.460056 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 15:12:50.460060 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:12:50.460077 (+ 17us) server_negotiation.cc:407] Connection header received
0807 15:12:50.460163 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:12:50.460166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:12:50.460244 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:12:50.460366 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:12:50.460639 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.461182 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.462078 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:12:50.462324 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:12:50.462438 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:12:50.462807 (+ 369us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:12:50.463258 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:12:50.463401 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 15:12:50.463463 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 15:12:50.463525 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 15:12:50.470526 6681 raft_consensus.cc:1261] T 19580d44f1a44a97a2818dee8deda7eb P 489972c8b9ef4a339aa8c09de646d108 [term 1 FOLLOWER]: Refusing update from remote peer caee2f5650aa4f1e8ae25fc770229752: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:12:50.470526 6295 raft_consensus.cc:1261] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 1 FOLLOWER]: Refusing update from remote peer caee2f5650aa4f1e8ae25fc770229752: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:12:50.471355 6789 consensus_queue.cc:1035] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752 [LEADER]: Connected to new peer: Peer: permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:12:50.471609 6794 consensus_queue.cc:1035] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:12:50.481091 6798 mvcc.cc:205] Tried to move back new op lower bound from 7315929376638365696 to 7315929376552894464. Current Snapshot: MvccSnapshot[applied={T|T < 7315929376638365696}]
I20260807 15:12:50.484030 6799 mvcc.cc:205] Tried to move back new op lower bound from 7315929376638365696 to 7315929376552894464. Current Snapshot: MvccSnapshot[applied={T|T < 7315929376638365696 or (T in {7315929376638365696})}]
I20260807 15:12:50.489626 6083 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:12:50.489830 6083 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:12:50.492384 6083 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67: Sending DeleteTablet for 3 replicas of tablet 19580d44f1a44a97a2818dee8deda7eb
I20260807 15:12:50.493299 6452 tablet_service.cc:1514] Processing DeleteTablet for tablet 19580d44f1a44a97a2818dee8deda7eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:12:50 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:50144
I20260807 15:12:50.493309 6267 tablet_service.cc:1514] Processing DeleteTablet for tablet 19580d44f1a44a97a2818dee8deda7eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:12:50 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:51746
I20260807 15:12:50.493945 6650 tablet_service.cc:1514] Processing DeleteTablet for tablet 19580d44f1a44a97a2818dee8deda7eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:12:50 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:39200
I20260807 15:12:50.494486 6812 tablet_replica.cc:330] T 19580d44f1a44a97a2818dee8deda7eb P 489972c8b9ef4a339aa8c09de646d108: stopping tablet replica
I20260807 15:12:50.494529 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 6175
I20260807 15:12:50.496964 6812 raft_consensus.cc:2229] T 19580d44f1a44a97a2818dee8deda7eb P 489972c8b9ef4a339aa8c09de646d108 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:12:50.497386 6812 raft_consensus.cc:2258] T 19580d44f1a44a97a2818dee8deda7eb P 489972c8b9ef4a339aa8c09de646d108 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:12:50.501078 6812 ts_tablet_manager.cc:1905] T 19580d44f1a44a97a2818dee8deda7eb P 489972c8b9ef4a339aa8c09de646d108: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:12:50.504742 6812 ts_tablet_manager.cc:1918] T 19580d44f1a44a97a2818dee8deda7eb P 489972c8b9ef4a339aa8c09de646d108: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:12:50.504911 6812 log.cc:1196] T 19580d44f1a44a97a2818dee8deda7eb P 489972c8b9ef4a339aa8c09de646d108: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/19580d44f1a44a97a2818dee8deda7eb
I20260807 15:12:50.505280 6812 ts_tablet_manager.cc:1939] T 19580d44f1a44a97a2818dee8deda7eb P 489972c8b9ef4a339aa8c09de646d108: Deleting consensus metadata
I20260807 15:12:50.508281 6058 catalog_manager.cc:4841] TS 489972c8b9ef4a339aa8c09de646d108 (127.15.226.67:46649): tablet 19580d44f1a44a97a2818dee8deda7eb (table test-table [id=e233f0e91c164988ae6db95711aca15c]) successfully deleted
I20260807 15:12:50.514161 6811 tablet_replica.cc:330] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752: stopping tablet replica
I20260807 15:12:50.514639 6811 raft_consensus.cc:2229] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:12:50.515138 6811 raft_consensus.cc:2258] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:12:50.523150 6811 ts_tablet_manager.cc:1905] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:12:50.525779 6811 ts_tablet_manager.cc:1918] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:12:50.525946 6811 log.cc:1196] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/19580d44f1a44a97a2818dee8deda7eb
I20260807 15:12:50.526300 6811 ts_tablet_manager.cc:1939] T 19580d44f1a44a97a2818dee8deda7eb P caee2f5650aa4f1e8ae25fc770229752: Deleting consensus metadata
I20260807 15:12:50.527688 6060 catalog_manager.cc:4841] TS caee2f5650aa4f1e8ae25fc770229752 (127.15.226.66:43769): tablet 19580d44f1a44a97a2818dee8deda7eb (table test-table [id=e233f0e91c164988ae6db95711aca15c]) successfully deleted
W20260807 15:12:50.528932 6059 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.226.65:36265 (error 108)
I20260807 15:12:50.529249 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 6367
I20260807 15:12:50.532270 6766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:12:50.531929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:36265 (local address 127.0.0.1:51760)
0807 15:12:50.532079 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 15:12:50.532162 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.65:36265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 15:12:50.532450 6059 catalog_manager.cc:4584] TS 3f1e4a721b8f4c6e8ff8325b0e48db50 (127.15.226.65:36265): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 19580d44f1a44a97a2818dee8deda7eb: Network error: Client connection negotiation failed: client connection to 127.15.226.65:36265: connect: Connection refused (error 111)
I20260807 15:12:50.542586 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 6561
I20260807 15:12:50.553565 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 6025
I20260807 15:13:10.571347 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:36145
--webserver_interface=127.15.226.126
--webserver_port=36381
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:44395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:36145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:13:10.694897 6815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:10.695225 6815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:10.695271 6815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:10.699592 6815 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 15:13:10.699748 6815 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 15:13:10.699783 6815 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 15:13:10.699811 6815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:13:10.699863 6815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:10.699905 6815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:10.699929 6815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:13:10.699951 6815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:13:10.705704 6815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:44395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:36145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:36145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=36381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:10.707010 6815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:10.708132 6815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:10.724354 6826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:10.724406 6825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:10.724447 6815 server_base.cc:1034] running on GCE node
W20260807 15:13:10.724354 6829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:10.725461 6815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:10.726869 6815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:10.728085 6815 hybrid_clock.cc:648] HybridClock initialized: now 1786115590728076 us; error 41 us; skew 500 ppm
Aug 07 15:13:10 dist-test-slave-3kk6 krb5kdc[6008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115590, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:10.734815 6815 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:13:10.735960 6815 webserver.cc:460] Webserver started at http://127.15.226.126:36381/ using document root <none> and password file <none>
I20260807 15:13:10.736585 6815 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:10.736670 6815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:10.741299 6815 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:13:10.743829 6836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:10.745337 6815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:13:10.745484 6815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "dc6bb68a43ef448383e50f723c502b67"
format_stamp: "Formatted at 2026-08-07 15:12:49 on dist-test-slave-3kk6"
server_key: "b80bb9a26f96c1d29ca3878ae9a0e7ca"
server_key_iv: "8f20d99e5b71f1281f1845bbc3ee10b0"
server_key_version: "encryptionkey@0"
I20260807 15:13:10.746230 6815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:10.774314 6815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:10.778137 6815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:10.778340 6815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:10.786439 6815 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:36145
I20260807 15:13:10.786842 6914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:36145 every 8 connection(s)
I20260807 15:13:10.787842 6815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:13:10.791849 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 6815
I20260807 15:13:10.792439 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:36265
--local_ip_for_outbound_sockets=127.15.226.65
--tserver_master_addrs=127.15.226.126:36145
--webserver_port=35051
--webserver_interface=127.15.226.65
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:44395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:13:10.793375 6915 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:13:10.795997 6915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67: Bootstrap starting.
I20260807 15:13:10.816227 6915 log.cc:824] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:10.826005 6915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:13:10.826550 6915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67: Bootstrap complete.
I20260807 15:13:10.831266 6915 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc6bb68a43ef448383e50f723c502b67" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36145 } }
I20260807 15:13:10.831790 6915 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc6bb68a43ef448383e50f723c502b67, State: Initialized, Role: FOLLOWER
I20260807 15:13:10.832330 6915 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc6bb68a43ef448383e50f723c502b67" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36145 } }
I20260807 15:13:10.832489 6915 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:13:10.832592 6915 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:13:10.832719 6915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 1 FOLLOWER]: Advancing to term 2
I20260807 15:13:10.834985 6915 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc6bb68a43ef448383e50f723c502b67" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36145 } }
I20260807 15:13:10.835366 6915 leader_election.cc:302] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc6bb68a43ef448383e50f723c502b67; no voters:
I20260807 15:13:10.835754 6915 leader_election.cc:288] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 15:13:10.835883 6922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 15:13:10.836089 6922 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 2 LEADER]: Becoming Leader. State: Replica: dc6bb68a43ef448383e50f723c502b67, State: Running, Role: LEADER
I20260807 15:13:10.836423 6922 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc6bb68a43ef448383e50f723c502b67" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36145 } }
I20260807 15:13:10.837239 6915 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:13:10.837742 6922 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc6bb68a43ef448383e50f723c502b67. Latest consensus state: current_term: 2 leader_uuid: "dc6bb68a43ef448383e50f723c502b67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc6bb68a43ef448383e50f723c502b67" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36145 } } }
I20260807 15:13:10.837880 6922 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [sys.catalog]: This master's current role is: LEADER
I20260807 15:13:10.838311 6930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:13:10.838337 6925 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "dc6bb68a43ef448383e50f723c502b67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc6bb68a43ef448383e50f723c502b67" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36145 } } }
I20260807 15:13:10.838596 6925 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [sys.catalog]: This master's current role is: LEADER
I20260807 15:13:10.844796 6930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:13:10.845532 6930 catalog_manager.cc:1259] Loaded cluster ID: 6eaf17e2d86d4117b916114ac1d0216c
I20260807 15:13:10.845763 6930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:13:10.849048 6930 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:13:10.850162 6930 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67: Loaded TSK: 0
I20260807 15:13:10.865764 6930 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67: Generated new TSK 1
I20260807 15:13:10.866107 6930 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:13:10.939085 6918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:10.939499 6918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:10.939600 6918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:10.944550 6918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:10.944689 6918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:10.944810 6918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:13:10.952073 6918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:44395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:36265
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=35051
--tserver_master_addrs=127.15.226.126:36145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:10.953338 6918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:10.954500 6918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:10.965828 6960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:10.965845 6958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:10.965827 6962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:10.966382 6918 server_base.cc:1034] running on GCE node
I20260807 15:13:10.967026 6918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:10.968148 6918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:10.969406 6918 hybrid_clock.cc:648] HybridClock initialized: now 1786115590969372 us; error 65 us; skew 500 ppm
Aug 07 15:13:10 dist-test-slave-3kk6 krb5kdc[6008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115590, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:10.973759 6918 init.cc:377] Logged in from keytab as kudu/127.15.226.65@KRBTEST.COM (short username kudu)
I20260807 15:13:10.975368 6918 webserver.cc:460] Webserver started at http://127.15.226.65:35051/ using document root <none> and password file <none>
I20260807 15:13:10.976076 6918 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:10.976228 6918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:10.981159 6918 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260807 15:13:10.984028 6970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:10.985664 6918 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:13:10.985878 6918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50"
format_stamp: "Formatted at 2026-08-07 15:12:49 on dist-test-slave-3kk6"
server_key: "b9dbd8a1676b948e52ec2ef492ddb05b"
server_key_iv: "c7b10e4a445bb65768d427d4393b99b9"
server_key_version: "encryptionkey@0"
I20260807 15:13:10.986515 6918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:11.000259 6918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:11.004580 6918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:11.004866 6918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:11.005693 6918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:11.007252 6983 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 15:13:11.012158 6918 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 15:13:11.012306 6918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260807 15:13:11.012418 6918 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 15:13:11.014535 6918 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 15:13:11.014632 6918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260807 15:13:11.015036 6983 tablet_bootstrap.cc:492] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50: Bootstrap starting.
I20260807 15:13:11.027287 6918 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:36265
I20260807 15:13:11.027360 7141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:36265 every 8 connection(s)
I20260807 15:13:11.028491 6918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:13:11.032557 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 6918
I20260807 15:13:11.033113 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:43769
--local_ip_for_outbound_sockets=127.15.226.66
--tserver_master_addrs=127.15.226.126:36145
--webserver_port=46525
--webserver_interface=127.15.226.66
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:44395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:13:11.036401 6983 log.cc:824] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50: Log is configured to *not* fsync() on all Append() calls
Aug 07 15:13:11 dist-test-slave-3kk6 krb5kdc[6008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115590, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:13:11.045469 6983 tablet_bootstrap.cc:492] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:13:11.046007 6983 tablet_bootstrap.cc:492] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50: Bootstrap complete.
I20260807 15:13:11.047708 6983 ts_tablet_manager.cc:1397] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50: Time spent bootstrapping tablet: real 0.033s user 0.017s sys 0.015s
I20260807 15:13:11.050814 7145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:11.030718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:59633 (local address 127.15.226.126:36145)
0807 15:13:11.031154 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:11.031175 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:11.031550 (+ 375us) server_negotiation.cc:407] Connection header received
0807 15:13:11.032826 (+ 1276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:11.032838 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:11.033165 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:11.033608 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:11.034964 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.036636 (+ 1672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.038818 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.039692 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.039919 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:11.043444 (+ 3525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:11.043456 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:11.043477 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:11.043528 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:11.046269 (+ 2741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:11.047142 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:11.047152 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:11.047161 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:11.047377 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:11.047955 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:11.047961 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:11.047963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:11.048531 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:11.048673 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:11.049381 (+ 708us) server_negotiation.cc:299] Negotiation successful
0807 15:13:11.049720 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":129,"threads_started":1}
I20260807 15:13:11.050832 7146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:11.030783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.65:59633)
0807 15:13:11.031334 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:11.031395 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:11.032338 (+ 943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:11.033677 (+ 1339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:11.033686 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:11.034178 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:11.034761 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:11.034780 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.036867 (+ 2087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.036874 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:11.038569 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:11.038588 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.039888 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.039892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:11.040099 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:11.040916 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:13:11.040939 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:11.043123 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:11.046455 (+ 3332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:11.046465 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:11.046483 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:11.046965 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:11.047509 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:11.047515 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:11.047520 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:11.047719 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:11.048681 (+ 962us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:11.048689 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:11.048998 (+ 309us) client_negotiation.cc:769] Sending connection context
0807 15:13:11.049203 (+ 205us) client_negotiation.cc:240] Negotiation successful
0807 15:13:11.049719 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":173,"threads_started":1}
I20260807 15:13:11.050983 6983 raft_consensus.cc:348] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } } peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } }
I20260807 15:13:11.051558 6983 raft_consensus.cc:729] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1e4a721b8f4c6e8ff8325b0e48db50, State: Initialized, Role: FOLLOWER
I20260807 15:13:11.052120 6983 consensus_queue.cc:260] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } } peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } }
I20260807 15:13:11.053182 6983 ts_tablet_manager.cc:1428] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260807 15:13:11.054087 7143 heartbeater.cc:347] Connected to a master server at 127.15.226.126:36145
I20260807 15:13:11.054371 7143 heartbeater.cc:464] Registering TS with master...
I20260807 15:13:11.055065 7143 heartbeater.cc:511] Master 127.15.226.126:36145 requested a full tablet report, sending...
I20260807 15:13:11.058333 6864 ts_manager.cc:194] Registered new tserver with Master: 3f1e4a721b8f4c6e8ff8325b0e48db50 (127.15.226.65:36265)
I20260807 15:13:11.063109 6864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.65@KRBTEST.COM'} at 127.15.226.65:59633
I20260807 15:13:11.066088 7143 heartbeater.cc:503] Master 127.15.226.126:36145 was elected leader, sending a full tablet report...
Aug 07 15:13:11 dist-test-slave-3kk6 krb5kdc[6008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115590, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for kudu/127.15.226.65@KRBTEST.COM
I20260807 15:13:11.075022 7152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:11.062553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54286 (local address 127.15.226.65:36265)
0807 15:13:11.063603 (+ 1050us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:11.063618 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:11.063649 (+ 31us) server_negotiation.cc:407] Connection header received
0807 15:13:11.063717 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:11.063721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:11.063890 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:11.064080 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:11.065085 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.066227 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.067392 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.067746 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.068027 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:11.070704 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:11.070716 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:11.070727 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:11.070752 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:11.072642 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:11.073260 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:11.073266 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:11.073267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:11.073359 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:11.073689 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:11.073694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:11.073695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:11.074000 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:11.074242 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:11.074593 (+ 351us) server_negotiation.cc:299] Negotiation successful
0807 15:13:11.074823 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":931,"thread_start_us":181,"threads_started":1}
I20260807 15:13:11.075022 7151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:11.062430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:36265 (local address 127.0.0.1:54286)
0807 15:13:11.063210 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:11.063269 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:11.063460 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:11.064232 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:11.064241 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:11.064767 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:11.064920 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:11.064929 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.066388 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.066392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:11.067254 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:11.067265 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.067887 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.067892 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:11.068021 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:11.068715 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:13:11.068727 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:11.070503 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:11.072791 (+ 2288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:11.072805 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:11.072811 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:11.073095 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:11.073476 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:11.073479 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:11.073481 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:11.073587 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:11.074136 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:11.074168 (+ 32us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:11.074337 (+ 169us) client_negotiation.cc:769] Sending connection context
0807 15:13:11.074500 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 15:13:11.074813 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":166,"threads_started":1}
I20260807 15:13:11.076258 7039 tablet_service.cc:1514] Processing DeleteTablet for tablet 19580d44f1a44a97a2818dee8deda7eb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:12:50 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:54286
I20260807 15:13:11.077270 7154 tablet_replica.cc:330] stopping tablet replica
I20260807 15:13:11.077659 7154 raft_consensus.cc:2229] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:13:11.077965 7154 raft_consensus.cc:2258] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:13:11.079257 7154 ts_tablet_manager.cc:1905] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:13:11.084937 7154 ts_tablet_manager.cc:1918] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:13:11.085100 7154 log.cc:1196] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/wal/wals/19580d44f1a44a97a2818dee8deda7eb
I20260807 15:13:11.085459 7154 ts_tablet_manager.cc:1939] T 19580d44f1a44a97a2818dee8deda7eb P 3f1e4a721b8f4c6e8ff8325b0e48db50: Deleting consensus metadata
I20260807 15:13:11.087904 6840 catalog_manager.cc:4841] TS 3f1e4a721b8f4c6e8ff8325b0e48db50 (127.15.226.65:36265): tablet 19580d44f1a44a97a2818dee8deda7eb (table test-table [id=e233f0e91c164988ae6db95711aca15c]) successfully deleted
W20260807 15:13:11.175422 7148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:11.175827 7148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:11.175877 7148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:11.180328 7148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:11.180426 7148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:11.180491 7148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:13:11.186383 7148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:44395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:43769
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=46525
--tserver_master_addrs=127.15.226.126:36145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:11.187709 7148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:11.188860 7148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:11.200997 7166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:11.200937 7167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:11.201028 7169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:11.201277 7148 server_base.cc:1034] running on GCE node
I20260807 15:13:11.202021 7148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:11.203051 7148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:11.204300 7148 hybrid_clock.cc:648] HybridClock initialized: now 1786115591204263 us; error 63 us; skew 500 ppm
Aug 07 15:13:11 dist-test-slave-3kk6 krb5kdc[6008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115591, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:11.208182 7148 init.cc:377] Logged in from keytab as kudu/127.15.226.66@KRBTEST.COM (short username kudu)
I20260807 15:13:11.209335 7148 webserver.cc:460] Webserver started at http://127.15.226.66:46525/ using document root <none> and password file <none>
I20260807 15:13:11.209949 7148 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:11.210035 7148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:11.214879 7148 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:13:11.217823 7180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:11.219604 7148 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 15:13:11.219801 7148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "caee2f5650aa4f1e8ae25fc770229752"
format_stamp: "Formatted at 2026-08-07 15:12:49 on dist-test-slave-3kk6"
server_key: "45b4eb4fcd13fda47a072f9ff244adf8"
server_key_iv: "8b162e95b7d5f2e0ddcb7adedac46454"
server_key_version: "encryptionkey@0"
I20260807 15:13:11.220690 7148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:11.239521 7148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:11.243440 7148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:11.243693 7148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:11.244432 7148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:11.245379 7148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:11.245430 7148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:11.245499 7148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:11.245533 7148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:11.256371 7148 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:43769
I20260807 15:13:11.256426 7369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:43769 every 8 connection(s)
I20260807 15:13:11.257627 7148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:13:11.263811 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 7148
I20260807 15:13:11.264349 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:46649
--local_ip_for_outbound_sockets=127.15.226.67
--tserver_master_addrs=127.15.226.126:36145
--webserver_port=43265
--webserver_interface=127.15.226.67
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:44395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 15:13:11 dist-test-slave-3kk6 krb5kdc[6008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115591, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:13:11.275084 7145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:11.260074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:52241 (local address 127.15.226.126:36145)
0807 15:13:11.260275 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:11.260280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:11.261274 (+ 994us) server_negotiation.cc:407] Connection header received
0807 15:13:11.262743 (+ 1469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:11.262746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:11.262802 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:11.262887 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:11.264346 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.265187 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.266505 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.266838 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.266961 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:11.270537 (+ 3576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:11.270540 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:11.270542 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:11.270574 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:11.272479 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:11.273113 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:11.273117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:11.273121 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:11.273212 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:11.273612 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:11.273616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:11.273617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:11.273932 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:11.274041 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:11.274626 (+ 585us) server_negotiation.cc:299] Negotiation successful
0807 15:13:11.274894 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 15:13:11.276034 7372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:11.260530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.66:52241)
0807 15:13:11.261098 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:11.261148 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:11.262457 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:11.263083 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:11.263090 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:11.263514 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:11.264158 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:11.264175 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.265354 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.265356 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:11.266351 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:11.266362 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.267032 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.267037 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:11.267242 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:11.268274 (+ 1032us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:13:11.268294 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:11.270293 (+ 1999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:11.272628 (+ 2335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:11.272635 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:11.272655 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:11.272985 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:11.273322 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:11.273325 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:11.273327 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:11.273502 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:11.274060 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:11.274068 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:11.274362 (+ 294us) client_negotiation.cc:769] Sending connection context
0807 15:13:11.274580 (+ 218us) client_negotiation.cc:240] Negotiation successful
0807 15:13:11.275073 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":128,"threads_started":1}
I20260807 15:13:11.277439 7370 heartbeater.cc:347] Connected to a master server at 127.15.226.126:36145
I20260807 15:13:11.277766 7370 heartbeater.cc:464] Registering TS with master...
I20260807 15:13:11.278529 7370 heartbeater.cc:511] Master 127.15.226.126:36145 requested a full tablet report, sending...
I20260807 15:13:11.279919 6864 ts_manager.cc:194] Registered new tserver with Master: caee2f5650aa4f1e8ae25fc770229752 (127.15.226.66:43769)
I20260807 15:13:11.280591 6864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.66@KRBTEST.COM'} at 127.15.226.66:52241
W20260807 15:13:11.395385 7374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:11.395704 7374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:11.395751 7374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:11.400210 7374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:11.400319 7374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:11.400386 7374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:13:11.406380 7374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:44395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:46649
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=43265
--tserver_master_addrs=127.15.226.126:36145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:11.407747 7374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:11.408967 7374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:11.420049 7381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:11.420137 7374 server_base.cc:1034] running on GCE node
W20260807 15:13:11.420049 7382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:11.420071 7387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:11.420835 7374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:11.421559 7374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:11.422735 7374 hybrid_clock.cc:648] HybridClock initialized: now 1786115591422703 us; error 56 us; skew 500 ppm
Aug 07 15:13:11 dist-test-slave-3kk6 krb5kdc[6008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115591, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:11.425931 7374 init.cc:377] Logged in from keytab as kudu/127.15.226.67@KRBTEST.COM (short username kudu)
I20260807 15:13:11.426976 7374 webserver.cc:460] Webserver started at http://127.15.226.67:43265/ using document root <none> and password file <none>
I20260807 15:13:11.427603 7374 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:11.427726 7374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:11.432054 7374 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:13:11.434376 7397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:11.435806 7374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:13:11.435978 7374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "489972c8b9ef4a339aa8c09de646d108"
format_stamp: "Formatted at 2026-08-07 15:12:50 on dist-test-slave-3kk6"
server_key: "b0b1cd8b3d93b9f51071529b3be3a8dc"
server_key_iv: "7c90822e7528e0766090101f0f863fad"
server_key_version: "encryptionkey@0"
I20260807 15:13:11.436641 7374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:11.455621 7374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:11.459471 7374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:11.459738 7374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:11.460449 7374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:11.461421 7374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:11.461508 7374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:11.461589 7374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:11.461633 7374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:11.471975 7374 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:46649
I20260807 15:13:11.471977 7570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:46649 every 8 connection(s)
I20260807 15:13:11.473366 7374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:13:11.474265 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 7374
I20260807 15:13:11.484958 7145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:11.475182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33722 (local address 127.15.226.126:36145)
0807 15:13:11.475367 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:11.475373 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:11.475637 (+ 264us) server_negotiation.cc:407] Connection header received
0807 15:13:11.475845 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:11.475848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:11.475904 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:11.475996 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:11.477222 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.478010 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.479038 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.479287 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.479405 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:11.481045 (+ 1640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:11.481049 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:11.481052 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:11.481080 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:11.482822 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:11.483422 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:11.483428 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:11.483433 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:11.483536 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:11.483866 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:11.483869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:11.483871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:11.484101 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:11.484355 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:11.484623 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 15:13:11.484762 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 15:13:11 dist-test-slave-3kk6 krb5kdc[6008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115591, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:13:11.491359 7578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:11.475782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:50297 (local address 127.15.226.126:36145)
0807 15:13:11.476059 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:11.476062 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:11.477887 (+ 1825us) server_negotiation.cc:407] Connection header received
0807 15:13:11.478978 (+ 1091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:11.478981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:11.479039 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:11.479152 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:11.480784 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.481356 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.483295 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.483605 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.483719 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:11.486872 (+ 3153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:11.486876 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:11.486879 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:11.486909 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:11.488611 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:11.489259 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:11.489262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:11.489264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:11.489360 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:11.489859 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:11.489863 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:11.489912 (+ 49us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:11.490189 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:11.490291 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:11.491047 (+ 756us) server_negotiation.cc:299] Negotiation successful
0807 15:13:11.491184 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":129,"threads_started":1}
I20260807 15:13:11.492451 7579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:11.476448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.67:50297)
0807 15:13:11.477698 (+ 1250us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:11.477761 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:11.478747 (+ 986us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:11.479456 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:11.479465 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:11.479990 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:11.480617 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:11.480633 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.481522 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.481526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:11.483165 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:11.483177 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.483771 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.483774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:11.484020 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:11.484713 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:13:11.484739 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:11.486628 (+ 1889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:11.488754 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:11.488762 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:11.488783 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:11.489112 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:11.489507 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:11.489510 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:11.489512 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:11.489705 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:11.490315 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:11.490322 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:11.490768 (+ 446us) client_negotiation.cc:769] Sending connection context
0807 15:13:11.490969 (+ 201us) client_negotiation.cc:240] Negotiation successful
0807 15:13:11.491344 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":990,"thread_start_us":778,"threads_started":1}
I20260807 15:13:11.493808 7572 heartbeater.cc:347] Connected to a master server at 127.15.226.126:36145
I20260807 15:13:11.494073 7572 heartbeater.cc:464] Registering TS with master...
I20260807 15:13:11.494601 7572 heartbeater.cc:511] Master 127.15.226.126:36145 requested a full tablet report, sending...
I20260807 15:13:11.495976 6864 ts_manager.cc:194] Registered new tserver with Master: 489972c8b9ef4a339aa8c09de646d108 (127.15.226.67:46649)
I20260807 15:13:11.496650 6864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.67@KRBTEST.COM'} at 127.15.226.67:50297
I20260807 15:13:11.499598 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:13:11.504774 7578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:11.500574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33736 (local address 127.15.226.126:36145)
0807 15:13:11.500761 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:11.500767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:11.500967 (+ 200us) server_negotiation.cc:407] Connection header received
0807 15:13:11.501192 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:11.501195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:11.501305 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:11.501426 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:13:11.501697 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.502238 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.503300 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.503560 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.503677 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:11.504051 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:13:11.504209 (+ 158us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 15:13:11.504612 (+ 403us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33736: token expired
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 15:13:11.504887 7578 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33736: token expired
W20260807 15:13:11.505363 6757 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.15.226.126:36145)
I20260807 15:13:11.514223 7578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:11.505975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33746 (local address 127.15.226.126:36145)
0807 15:13:11.506116 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:11.506120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:11.506136 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:13:11.506265 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:11.506268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:11.506317 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:11.506407 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:11.507212 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.507743 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.508694 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.508931 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.509031 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:11.510549 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:11.510552 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:11.510554 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:11.510581 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:11.512426 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:11.512918 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:11.512920 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:11.512921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:11.513009 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:11.513327 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:11.513329 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:11.513330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:11.513522 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:11.513626 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:11.513883 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 15:13:11.513999 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:13:11.517051 6864 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33746:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:13:11.520578 6864 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:13:11.529613 7039 tablet_service.cc:1467] Processing CreateTablet for tablet fb92990b3a534ba09079d73b35136d19 (DEFAULT_TABLE table=test-table [id=a096815c570c40f1af76d5bc1eb94508]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:13:11.530421 7039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb92990b3a534ba09079d73b35136d19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:11.533865 6983 tablet_bootstrap.cc:492] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50: Bootstrap starting.
I20260807 15:13:11.534443 7151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:11.528358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:46649 (local address 127.0.0.1:37928)
0807 15:13:11.528733 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:11.528751 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:11.528845 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:11.529955 (+ 1110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:11.529959 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:11.529978 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:11.530133 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:11.530143 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.531562 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.531566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:11.532675 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:11.532688 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.533921 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.533928 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:11.534081 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:11.534137 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 15:13:11.534195 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 15:13:11.534298 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280}
I20260807 15:13:11.534706 7585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:11.528689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:43769 (local address 127.0.0.1:54744)
0807 15:13:11.529061 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:11.529076 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:11.529160 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:11.530329 (+ 1169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:11.530332 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:11.530351 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:11.530481 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:11.530490 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.532130 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.532134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:11.533118 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:11.533127 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.534245 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.534249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:11.534360 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:11.534398 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 15:13:11.534467 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 15:13:11.534541 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":157,"threads_started":1}
I20260807 15:13:11.535010 6983 tablet_bootstrap.cc:654] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:11.535300 7586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:11.528730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37928 (local address 127.15.226.67:46649)
0807 15:13:11.529369 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:11.529387 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:11.529419 (+ 32us) server_negotiation.cc:407] Connection header received
0807 15:13:11.529522 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:11.529527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:11.529783 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:11.529973 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:11.530265 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.531434 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.532837 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.533755 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.534780 (+ 1025us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:11.534981 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:11.535038 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 15:13:11.535151 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":167,"threads_started":1}
I20260807 15:13:11.535655 7587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:11.529236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54744 (local address 127.15.226.66:43769)
0807 15:13:11.529633 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:11.529656 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:11.529683 (+ 27us) server_negotiation.cc:407] Connection header received
0807 15:13:11.529775 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:11.529780 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:11.530154 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:11.530640 (+ 486us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:11.530885 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.531988 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.533254 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.534092 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.535148 (+ 1056us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:11.535319 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:11.535381 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 15:13:11.535492 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":149,"threads_started":1}
I20260807 15:13:11.536713 6983 tablet_bootstrap.cc:492] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50: No bootstrap required, opened a new log
I20260807 15:13:11.536840 6983 ts_tablet_manager.cc:1397] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 15:13:11.537345 6983 raft_consensus.cc:348] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } } peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } }
I20260807 15:13:11.537490 6983 raft_consensus.cc:374] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:11.537532 6983 raft_consensus.cc:729] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1e4a721b8f4c6e8ff8325b0e48db50, State: Initialized, Role: FOLLOWER
I20260807 15:13:11.537673 6983 consensus_queue.cc:260] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } } peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } }
I20260807 15:13:11.537984 6983 ts_tablet_manager.cc:1428] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 15:13:11.538731 7471 tablet_service.cc:1467] Processing CreateTablet for tablet fb92990b3a534ba09079d73b35136d19 (DEFAULT_TABLE table=test-table [id=a096815c570c40f1af76d5bc1eb94508]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:13:11.538987 7263 tablet_service.cc:1467] Processing CreateTablet for tablet fb92990b3a534ba09079d73b35136d19 (DEFAULT_TABLE table=test-table [id=a096815c570c40f1af76d5bc1eb94508]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:13:11.539955 7471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb92990b3a534ba09079d73b35136d19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:11.540118 7263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb92990b3a534ba09079d73b35136d19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:11.548211 7589 tablet_bootstrap.cc:492] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108: Bootstrap starting.
I20260807 15:13:11.548461 7590 tablet_bootstrap.cc:492] T fb92990b3a534ba09079d73b35136d19 P caee2f5650aa4f1e8ae25fc770229752: Bootstrap starting.
I20260807 15:13:11.551151 7589 tablet_bootstrap.cc:654] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:11.551620 7590 tablet_bootstrap.cc:654] T fb92990b3a534ba09079d73b35136d19 P caee2f5650aa4f1e8ae25fc770229752: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:11.552475 7590 log.cc:824] T fb92990b3a534ba09079d73b35136d19 P caee2f5650aa4f1e8ae25fc770229752: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:11.552520 7589 log.cc:824] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:11.554878 7590 tablet_bootstrap.cc:492] T fb92990b3a534ba09079d73b35136d19 P caee2f5650aa4f1e8ae25fc770229752: No bootstrap required, opened a new log
I20260807 15:13:11.554978 7589 tablet_bootstrap.cc:492] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108: No bootstrap required, opened a new log
I20260807 15:13:11.555127 7590 ts_tablet_manager.cc:1397] T fb92990b3a534ba09079d73b35136d19 P caee2f5650aa4f1e8ae25fc770229752: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 15:13:11.555190 7589 ts_tablet_manager.cc:1397] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 15:13:11.558314 7590 raft_consensus.cc:348] T fb92990b3a534ba09079d73b35136d19 P caee2f5650aa4f1e8ae25fc770229752 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } } peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } }
I20260807 15:13:11.558315 7589 raft_consensus.cc:348] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } } peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } }
I20260807 15:13:11.558635 7590 raft_consensus.cc:374] T fb92990b3a534ba09079d73b35136d19 P caee2f5650aa4f1e8ae25fc770229752 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:11.558629 7589 raft_consensus.cc:374] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:11.558736 7589 raft_consensus.cc:729] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 489972c8b9ef4a339aa8c09de646d108, State: Initialized, Role: FOLLOWER
I20260807 15:13:11.558758 7590 raft_consensus.cc:729] T fb92990b3a534ba09079d73b35136d19 P caee2f5650aa4f1e8ae25fc770229752 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caee2f5650aa4f1e8ae25fc770229752, State: Initialized, Role: FOLLOWER
I20260807 15:13:11.559264 7590 consensus_queue.cc:260] T fb92990b3a534ba09079d73b35136d19 P caee2f5650aa4f1e8ae25fc770229752 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } } peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } }
I20260807 15:13:11.559264 7589 consensus_queue.cc:260] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } } peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } }
I20260807 15:13:11.560454 7590 ts_tablet_manager.cc:1428] T fb92990b3a534ba09079d73b35136d19 P caee2f5650aa4f1e8ae25fc770229752: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 15:13:11.560457 7589 ts_tablet_manager.cc:1428] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 15:13:11.560590 7370 heartbeater.cc:503] Master 127.15.226.126:36145 was elected leader, sending a full tablet report...
I20260807 15:13:11.560796 7572 heartbeater.cc:503] Master 127.15.226.126:36145 was elected leader, sending a full tablet report...
I20260807 15:13:11.605374 7596 raft_consensus.cc:482] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:13:11.605618 7596 raft_consensus.cc:504] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } } peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } }
I20260807 15:13:11.606899 7596 leader_election.cc:288] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers caee2f5650aa4f1e8ae25fc770229752 (127.15.226.66:43769), 3f1e4a721b8f4c6e8ff8325b0e48db50 (127.15.226.65:36265)
I20260807 15:13:11.611577 7587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:11.607457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:41783 (local address 127.15.226.66:43769)
0807 15:13:11.607635 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:11.607640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:11.607663 (+ 23us) server_negotiation.cc:407] Connection header received
0807 15:13:11.607715 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:11.607717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:11.607760 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:11.607890 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:11.608323 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.609050 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.610345 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.611012 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.611140 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:11.611187 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:11.611403 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 15:13:11.611464 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 15:13:11.611603 7579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:11.607202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:43769 (local address 127.15.226.67:41783)
0807 15:13:11.607369 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:11.607390 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:11.607518 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:11.607934 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:11.607942 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:11.607965 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:11.608156 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:11.608165 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.609176 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.609179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:11.610096 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:11.610106 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.611142 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.611146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:11.611262 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:11.611290 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 15:13:11.611388 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 15:13:11.611469 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 15:13:11.612426 7597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:11.607317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:58687 (local address 127.15.226.65:36265)
0807 15:13:11.607656 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:11.607662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:11.608068 (+ 406us) server_negotiation.cc:407] Connection header received
0807 15:13:11.608173 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:11.608176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:11.608265 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:11.608379 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:11.608741 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.609374 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.611123 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.611842 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.611995 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:11.612190 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:11.612230 (+ 40us) server_negotiation.cc:299] Negotiation successful
0807 15:13:11.612303 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":125,"threads_started":1}
I20260807 15:13:11.612447 7598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:11.607202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:36265 (local address 127.15.226.67:58687)
0807 15:13:11.607956 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:11.607971 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:11.608082 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:11.608429 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:11.608439 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:11.608465 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:11.608600 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:11.608607 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.609495 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.609497 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:11.610959 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:11.610972 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.612015 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.612019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:11.612110 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:11.612131 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 15:13:11.612195 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 15:13:11.612269 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"mutex_wait_us":332,"thread_start_us":173,"threads_started":1}
I20260807 15:13:11.612438 7292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb92990b3a534ba09079d73b35136d19" candidate_uuid: "489972c8b9ef4a339aa8c09de646d108" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caee2f5650aa4f1e8ae25fc770229752" is_pre_election: true
I20260807 15:13:11.612882 7292 raft_consensus.cc:2454] T fb92990b3a534ba09079d73b35136d19 P caee2f5650aa4f1e8ae25fc770229752 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 489972c8b9ef4a339aa8c09de646d108 in term 0.
I20260807 15:13:11.613060 7070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb92990b3a534ba09079d73b35136d19" candidate_uuid: "489972c8b9ef4a339aa8c09de646d108" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" is_pre_election: true
I20260807 15:13:11.613415 7070 raft_consensus.cc:2454] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 489972c8b9ef4a339aa8c09de646d108 in term 0.
I20260807 15:13:11.613488 7401 leader_election.cc:302] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 489972c8b9ef4a339aa8c09de646d108, caee2f5650aa4f1e8ae25fc770229752; no voters:
I20260807 15:13:11.613924 7596 raft_consensus.cc:2781] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:13:11.614151 7596 raft_consensus.cc:482] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:13:11.614285 7596 raft_consensus.cc:3037] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:11.615634 7596 raft_consensus.cc:504] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } } peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } }
I20260807 15:13:11.616060 7596 leader_election.cc:288] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108 [CANDIDATE]: Term 1 election: Requested vote from peers caee2f5650aa4f1e8ae25fc770229752 (127.15.226.66:43769), 3f1e4a721b8f4c6e8ff8325b0e48db50 (127.15.226.65:36265)
I20260807 15:13:11.616506 7070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb92990b3a534ba09079d73b35136d19" candidate_uuid: "489972c8b9ef4a339aa8c09de646d108" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50"
I20260807 15:13:11.616669 7070 raft_consensus.cc:3037] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:11.618068 7070 raft_consensus.cc:2454] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 489972c8b9ef4a339aa8c09de646d108 in term 1.
I20260807 15:13:11.618379 7292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb92990b3a534ba09079d73b35136d19" candidate_uuid: "489972c8b9ef4a339aa8c09de646d108" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caee2f5650aa4f1e8ae25fc770229752"
I20260807 15:13:11.618584 7292 raft_consensus.cc:3037] T fb92990b3a534ba09079d73b35136d19 P caee2f5650aa4f1e8ae25fc770229752 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:11.618705 7400 leader_election.cc:302] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f1e4a721b8f4c6e8ff8325b0e48db50, 489972c8b9ef4a339aa8c09de646d108; no voters:
I20260807 15:13:11.618899 7596 raft_consensus.cc:2781] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:13:11.619221 7596 raft_consensus.cc:686] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108 [term 1 LEADER]: Becoming Leader. State: Replica: 489972c8b9ef4a339aa8c09de646d108, State: Running, Role: LEADER
I20260807 15:13:11.619616 7596 consensus_queue.cc:237] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } } peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } }
I20260807 15:13:11.619951 7292 raft_consensus.cc:2454] T fb92990b3a534ba09079d73b35136d19 P caee2f5650aa4f1e8ae25fc770229752 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 489972c8b9ef4a339aa8c09de646d108 in term 1.
I20260807 15:13:11.623433 6864 catalog_manager.cc:5495] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108 reported cstate change: term changed from 0 to 1, leader changed from <none> to 489972c8b9ef4a339aa8c09de646d108 (127.15.226.67). New cstate: current_term: 1 leader_uuid: "489972c8b9ef4a339aa8c09de646d108" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } health_report { overall_health: UNKNOWN } } }
I20260807 15:13:11.646835 7586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:11.643428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37938 (local address 127.15.226.67:46649)
0807 15:13:11.643565 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:11.643569 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:11.643583 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:13:11.643663 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:11.643665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:11.643721 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:11.643815 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:13:11.644038 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.644552 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.645346 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:11.645568 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:11.645645 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:11.645963 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:13:11.646398 (+ 435us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:13:11.646494 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:11.646557 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 15:13:11.646612 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:13:11.653491 7070 raft_consensus.cc:1261] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 1 FOLLOWER]: Refusing update from remote peer 489972c8b9ef4a339aa8c09de646d108: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:13:11.653491 7292 raft_consensus.cc:1261] T fb92990b3a534ba09079d73b35136d19 P caee2f5650aa4f1e8ae25fc770229752 [term 1 FOLLOWER]: Refusing update from remote peer 489972c8b9ef4a339aa8c09de646d108: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:13:11.654459 7596 consensus_queue.cc:1035] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108 [LEADER]: Connected to new peer: Peer: permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:13:11.654799 7599 consensus_queue.cc:1035] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:13:11.661140 7603 mvcc.cc:205] Tried to move back new op lower bound from 7315929463403859968 to 7315929463277568000. Current Snapshot: MvccSnapshot[applied={T|T < 7315929463403859968}]
I20260807 15:13:11.661298 7606 mvcc.cc:205] Tried to move back new op lower bound from 7315929463403859968 to 7315929463277568000. Current Snapshot: MvccSnapshot[applied={T|T < 7315929463403859968}]
I20260807 15:13:11.670843 6864 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:13:11.671058 6864 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:13:11.673673 6864 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67: Sending DeleteTablet for 3 replicas of tablet fb92990b3a534ba09079d73b35136d19
I20260807 15:13:11.674536 7039 tablet_service.cc:1514] Processing DeleteTablet for tablet fb92990b3a534ba09079d73b35136d19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:13:11 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:54286
I20260807 15:13:11.674521 7471 tablet_service.cc:1514] Processing DeleteTablet for tablet fb92990b3a534ba09079d73b35136d19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:13:11 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:37928
I20260807 15:13:11.674583 7263 tablet_service.cc:1514] Processing DeleteTablet for tablet fb92990b3a534ba09079d73b35136d19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:13:11 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:54744
I20260807 15:13:11.675269 7617 tablet_replica.cc:330] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50: stopping tablet replica
I20260807 15:13:11.675387 7619 tablet_replica.cc:330] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108: stopping tablet replica
I20260807 15:13:11.675384 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 6918
I20260807 15:13:11.675444 7617 raft_consensus.cc:2229] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:13:11.675616 7617 raft_consensus.cc:2258] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:13:11.681099 7619 raft_consensus.cc:2229] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:13:11.681644 7619 raft_consensus.cc:2258] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:13:11.683327 7618 tablet_replica.cc:330] T fb92990b3a534ba09079d73b35136d19 P caee2f5650aa4f1e8ae25fc770229752: stopping tablet replica
I20260807 15:13:11.683883 7618 raft_consensus.cc:2229] T fb92990b3a534ba09079d73b35136d19 P caee2f5650aa4f1e8ae25fc770229752 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:13:11.684371 7618 raft_consensus.cc:2258] T fb92990b3a534ba09079d73b35136d19 P caee2f5650aa4f1e8ae25fc770229752 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:13:11.688666 7618 ts_tablet_manager.cc:1905] T fb92990b3a534ba09079d73b35136d19 P caee2f5650aa4f1e8ae25fc770229752: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:13:11.692381 7618 ts_tablet_manager.cc:1918] T fb92990b3a534ba09079d73b35136d19 P caee2f5650aa4f1e8ae25fc770229752: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:13:11.692507 7618 log.cc:1196] T fb92990b3a534ba09079d73b35136d19 P caee2f5650aa4f1e8ae25fc770229752: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/fb92990b3a534ba09079d73b35136d19
I20260807 15:13:11.692869 7618 ts_tablet_manager.cc:1939] T fb92990b3a534ba09079d73b35136d19 P caee2f5650aa4f1e8ae25fc770229752: Deleting consensus metadata
I20260807 15:13:11.694387 6842 catalog_manager.cc:4841] TS caee2f5650aa4f1e8ae25fc770229752 (127.15.226.66:43769): tablet fb92990b3a534ba09079d73b35136d19 (table test-table [id=a096815c570c40f1af76d5bc1eb94508]) successfully deleted
I20260807 15:13:11.702498 7619 ts_tablet_manager.cc:1905] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:13:11.705308 7619 ts_tablet_manager.cc:1918] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:13:11.705473 7619 log.cc:1196] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/fb92990b3a534ba09079d73b35136d19
I20260807 15:13:11.705852 7619 ts_tablet_manager.cc:1939] T fb92990b3a534ba09079d73b35136d19 P 489972c8b9ef4a339aa8c09de646d108: Deleting consensus metadata
I20260807 15:13:11.707399 6837 catalog_manager.cc:4841] TS 489972c8b9ef4a339aa8c09de646d108 (127.15.226.67:46649): tablet fb92990b3a534ba09079d73b35136d19 (table test-table [id=a096815c570c40f1af76d5bc1eb94508]) successfully deleted
W20260807 15:13:11.709223 6840 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.226.65:36265 (error 108)
I20260807 15:13:11.710476 7585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:11.710108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:36265 (local address 127.0.0.1:54294)
0807 15:13:11.710298 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:11.710369 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.65:36265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260807 15:13:11.711015 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 7148
W20260807 15:13:11.710701 6840 catalog_manager.cc:4584] TS 3f1e4a721b8f4c6e8ff8325b0e48db50 (127.15.226.65:36265): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fb92990b3a534ba09079d73b35136d19: Network error: Client connection negotiation failed: client connection to 127.15.226.65:36265: connect: Connection refused (error 111)
I20260807 15:13:11.728114 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 7374
I20260807 15:13:11.745839 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 6815
I20260807 15:13:31.761118 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:36145
--webserver_interface=127.15.226.126
--webserver_port=36381
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:44395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:36145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:13:31.881461 7621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:31.881867 7621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:31.881959 7621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:31.886193 7621 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 15:13:31.886324 7621 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 15:13:31.886351 7621 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 15:13:31.886379 7621 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:13:31.886422 7621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:31.886484 7621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:31.886523 7621 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:13:31.886557 7621 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:13:31.892388 7621 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:44395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:36145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:36145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=36381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:31.893667 7621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:31.894850 7621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:31.904619 7629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:31.904587 7626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:31.904629 7621 server_base.cc:1034] running on GCE node
W20260807 15:13:31.904587 7627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:31.905642 7621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:31.906905 7621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:31.908102 7621 hybrid_clock.cc:648] HybridClock initialized: now 1786115611908068 us; error 54 us; skew 500 ppm
Aug 07 15:13:31 dist-test-slave-3kk6 krb5kdc[6008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115611, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:31.916579 7621 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:13:31.917622 7621 webserver.cc:460] Webserver started at http://127.15.226.126:36381/ using document root <none> and password file <none>
I20260807 15:13:31.918251 7621 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:31.918381 7621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:31.922544 7621 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:13:31.924993 7640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:31.926350 7621 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:13:31.926527 7621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "dc6bb68a43ef448383e50f723c502b67"
format_stamp: "Formatted at 2026-08-07 15:12:49 on dist-test-slave-3kk6"
server_key: "b80bb9a26f96c1d29ca3878ae9a0e7ca"
server_key_iv: "8f20d99e5b71f1281f1845bbc3ee10b0"
server_key_version: "encryptionkey@0"
I20260807 15:13:31.927296 7621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:31.948678 7621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:31.952606 7621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:31.952888 7621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:31.961225 7621 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:36145
I20260807 15:13:31.961544 7712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:36145 every 8 connection(s)
I20260807 15:13:31.962525 7621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:13:31.967391 7713 sys_catalog.cc:263] Verifying existing consensus state
I20260807 15:13:31.969390 7713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67: Bootstrap starting.
I20260807 15:13:31.970374 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 7621
I20260807 15:13:31.970994 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:36265
--local_ip_for_outbound_sockets=127.15.226.65
--tserver_master_addrs=127.15.226.126:36145
--webserver_port=35051
--webserver_interface=127.15.226.65
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:44395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:13:31.991714 7713 log.cc:824] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:32.004254 7713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:13:32.004762 7713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67: Bootstrap complete.
I20260807 15:13:32.009651 7713 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc6bb68a43ef448383e50f723c502b67" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36145 } }
I20260807 15:13:32.010128 7713 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc6bb68a43ef448383e50f723c502b67, State: Initialized, Role: FOLLOWER
I20260807 15:13:32.010769 7713 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc6bb68a43ef448383e50f723c502b67" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36145 } }
I20260807 15:13:32.010922 7713 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:13:32.011021 7713 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:13:32.011159 7713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 2 FOLLOWER]: Advancing to term 3
I20260807 15:13:32.013403 7713 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc6bb68a43ef448383e50f723c502b67" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36145 } }
I20260807 15:13:32.013790 7713 leader_election.cc:302] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc6bb68a43ef448383e50f723c502b67; no voters:
I20260807 15:13:32.014197 7713 leader_election.cc:288] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 15:13:32.014345 7720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 15:13:32.014585 7720 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [term 3 LEADER]: Becoming Leader. State: Replica: dc6bb68a43ef448383e50f723c502b67, State: Running, Role: LEADER
I20260807 15:13:32.014935 7720 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc6bb68a43ef448383e50f723c502b67" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36145 } }
I20260807 15:13:32.015623 7713 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:13:32.016027 7720 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc6bb68a43ef448383e50f723c502b67. Latest consensus state: current_term: 3 leader_uuid: "dc6bb68a43ef448383e50f723c502b67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc6bb68a43ef448383e50f723c502b67" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36145 } } }
I20260807 15:13:32.016165 7720 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [sys.catalog]: This master's current role is: LEADER
I20260807 15:13:32.016544 7721 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "dc6bb68a43ef448383e50f723c502b67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc6bb68a43ef448383e50f723c502b67" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36145 } } }
I20260807 15:13:32.016742 7721 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67 [sys.catalog]: This master's current role is: LEADER
I20260807 15:13:32.016655 7724 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:13:32.022997 7724 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:13:32.023792 7724 catalog_manager.cc:1259] Loaded cluster ID: 6eaf17e2d86d4117b916114ac1d0216c
I20260807 15:13:32.023866 7724 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:13:32.026765 7724 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:13:32.027637 7724 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67: Loaded TSK: 0
I20260807 15:13:32.027734 7724 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dc6bb68a43ef448383e50f723c502b67: Loaded TSK: 1
F20260807 15:13:32.028596 7724 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f979e40a9ad google::LogMessage::Fail() at ??:0
@ 0x7f979e40d88a google::LogMessage::SendToLog() at ??:0
@ 0x7f979e40a4de google::LogMessage::Flush() at ??:0
@ 0x7f979e40ba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f97a10ce18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f97a10c6f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f97a8d6f7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f97a8d4999c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f97a8d4a178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f97a8d8bdf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f97a8daa82b std::function<>::operator()() at ??:0
@ 0x7f97a8d49b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f97a8d4af3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f97a8d47579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f97a8d8b640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f97a065c3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f97a065cc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f97a065e3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f97a064d512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f97a1c916db start_thread at ??:0
@ 0x7f979cf0861f clone at ??:0
W20260807 15:13:32.116055 7716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:32.116479 7716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:32.116559 7716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:32.123292 7716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:32.123378 7716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:32.123481 7716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:13:32.132396 7716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:44395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:36265
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=35051
--tserver_master_addrs=127.15.226.126:36145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:32.133977 7716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:32.135485 7716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:32.149135 7786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:32.149251 7789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:32.149570 7787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:32.150218 7716 server_base.cc:1034] running on GCE node
I20260807 15:13:32.150897 7716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:32.151891 7716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:32.153105 7716 hybrid_clock.cc:648] HybridClock initialized: now 1786115612153089 us; error 35 us; skew 500 ppm
Aug 07 15:13:32 dist-test-slave-3kk6 krb5kdc[6008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115612, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:32.156546 7716 init.cc:377] Logged in from keytab as kudu/127.15.226.65@KRBTEST.COM (short username kudu)
I20260807 15:13:32.157650 7716 webserver.cc:460] Webserver started at http://127.15.226.65:35051/ using document root <none> and password file <none>
I20260807 15:13:32.158316 7716 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:32.158390 7716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:32.163532 7716 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:13:32.166157 7800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:32.167717 7716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:13:32.167901 7716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50"
format_stamp: "Formatted at 2026-08-07 15:12:49 on dist-test-slave-3kk6"
server_key: "b9dbd8a1676b948e52ec2ef492ddb05b"
server_key_iv: "c7b10e4a445bb65768d427d4393b99b9"
server_key_version: "encryptionkey@0"
I20260807 15:13:32.168563 7716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:32.191198 7716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:32.195103 7716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:32.195358 7716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:32.196041 7716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:32.197360 7815 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 15:13:32.203516 7716 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 15:13:32.203655 7716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260807 15:13:32.203752 7716 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 15:13:32.205832 7716 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 15:13:32.205928 7716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260807 15:13:32.206274 7815 tablet_bootstrap.cc:492] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50: Bootstrap starting.
I20260807 15:13:32.222330 7716 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:36265
I20260807 15:13:32.222711 7967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:36265 every 8 connection(s)
I20260807 15:13:32.223744 7716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:13:32.230947 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 7716
I20260807 15:13:32.231451 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:43769
--local_ip_for_outbound_sockets=127.15.226.66
--tserver_master_addrs=127.15.226.126:36145
--webserver_port=46525
--webserver_interface=127.15.226.66
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:44395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:13:32.234238 7815 log.cc:824] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:32.242003 7815 tablet_bootstrap.cc:492] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 15:13:32.242460 7815 tablet_bootstrap.cc:492] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50: Bootstrap complete.
I20260807 15:13:32.243865 7815 ts_tablet_manager.cc:1397] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50: Time spent bootstrapping tablet: real 0.038s user 0.022s sys 0.011s
I20260807 15:13:32.246484 7815 raft_consensus.cc:348] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } } peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } }
I20260807 15:13:32.247044 7815 raft_consensus.cc:729] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1e4a721b8f4c6e8ff8325b0e48db50, State: Initialized, Role: FOLLOWER
I20260807 15:13:32.247617 7815 consensus_queue.cc:260] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } } peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } }
I20260807 15:13:32.248592 7815 ts_tablet_manager.cc:1428] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
W20260807 15:13:32.324918 7972 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:13:32.226362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.65:42993)
0807 15:13:32.226937 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:32.226998 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:32.227905 (+ 907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:32.324068 (+ 96163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":175,"threads_started":1}
W20260807 15:13:32.325618 7970 heartbeater.cc:644] Failed to heartbeat to 127.15.226.126:36145 (0 consecutive failures): Network error: Failed to ping master at 127.15.226.126:36145: Client connection negotiation failed: client connection to 127.15.226.126:36145: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 15:13:32.326465 7972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:32.326174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.65:37387)
0807 15:13:32.326315 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:32.326403 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 15:13:32.327265 7972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:32.327041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.65:45247)
0807 15:13:32.327155 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:32.327205 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 15:13:32.327522 7970 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 15:13:32.372000 7974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:32.372393 7974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:32.372489 7974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:32.376864 7974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:32.376979 7974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:32.377094 7974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:13:32.383052 7974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:44395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:43769
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=46525
--tserver_master_addrs=127.15.226.126:36145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:32.384274 7974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:32.385370 7974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:32.396345 7984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:32.396490 7974 server_base.cc:1034] running on GCE node
W20260807 15:13:32.396298 7987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:32.396313 7983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:32.397205 7974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:32.397991 7974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:32.399215 7974 hybrid_clock.cc:648] HybridClock initialized: now 1786115612399193 us; error 34 us; skew 500 ppm
Aug 07 15:13:32 dist-test-slave-3kk6 krb5kdc[6008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115612, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:32.402819 7974 init.cc:377] Logged in from keytab as kudu/127.15.226.66@KRBTEST.COM (short username kudu)
I20260807 15:13:32.404150 7974 webserver.cc:460] Webserver started at http://127.15.226.66:46525/ using document root <none> and password file <none>
I20260807 15:13:32.404778 7974 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:32.404852 7974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:32.409320 7974 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:13:32.411868 7998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:32.413384 7974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:13:32.413556 7974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "caee2f5650aa4f1e8ae25fc770229752"
format_stamp: "Formatted at 2026-08-07 15:12:49 on dist-test-slave-3kk6"
server_key: "45b4eb4fcd13fda47a072f9ff244adf8"
server_key_iv: "8b162e95b7d5f2e0ddcb7adedac46454"
server_key_version: "encryptionkey@0"
I20260807 15:13:32.414175 7974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:32.427968 7974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:32.431505 7974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:32.431751 7974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:32.432406 7974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:32.433310 7974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:32.433385 7974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:32.433461 7974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:32.433506 7974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:32.443912 7974 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:43769
I20260807 15:13:32.443933 8155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:43769 every 8 connection(s)
I20260807 15:13:32.445084 7974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:13:32.449389 8159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:32.447448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.66:59129)
0807 15:13:32.447992 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:32.448114 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":121,"threads_started":1}
W20260807 15:13:32.449874 8156 heartbeater.cc:644] Failed to heartbeat to 127.15.226.126:36145 (0 consecutive failures): Network error: Failed to ping master at 127.15.226.126:36145: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
I20260807 15:13:32.450525 8159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:32.450299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.66:46693)
0807 15:13:32.450413 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:32.450467 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 15:13:32.451258 8159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:32.450989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.66:47921)
0807 15:13:32.451153 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:32.451203 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 15:13:32.451435 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 7974
W20260807 15:13:32.451489 8156 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 15:13:32.451936 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:46649
--local_ip_for_outbound_sockets=127.15.226.67
--tserver_master_addrs=127.15.226.126:36145
--webserver_port=43265
--webserver_interface=127.15.226.67
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:44395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:13:32.576870 8160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:32.577203 8160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:32.577283 8160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:32.581595 8160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:32.581677 8160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:32.581789 8160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:13:32.587532 8160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:44395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:46649
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=43265
--tserver_master_addrs=127.15.226.126:36145
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:32.588811 8160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:32.590020 8160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:32.600579 8176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:32.600557 8173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:32.600584 8172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:32.600958 8160 server_base.cc:1034] running on GCE node
I20260807 15:13:32.601425 8160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:32.602149 8160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:32.603334 8160 hybrid_clock.cc:648] HybridClock initialized: now 1786115612603299 us; error 54 us; skew 500 ppm
Aug 07 15:13:32 dist-test-slave-3kk6 krb5kdc[6008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115612, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:32.606441 8160 init.cc:377] Logged in from keytab as kudu/127.15.226.67@KRBTEST.COM (short username kudu)
I20260807 15:13:32.607499 8160 webserver.cc:460] Webserver started at http://127.15.226.67:43265/ using document root <none> and password file <none>
I20260807 15:13:32.608112 8160 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:32.608209 8160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:32.612434 8160 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260807 15:13:32.614820 8184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:32.616114 8160 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 15:13:32.616264 8160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "489972c8b9ef4a339aa8c09de646d108"
format_stamp: "Formatted at 2026-08-07 15:12:50 on dist-test-slave-3kk6"
server_key: "b0b1cd8b3d93b9f51071529b3be3a8dc"
server_key_iv: "7c90822e7528e0766090101f0f863fad"
server_key_version: "encryptionkey@0"
I20260807 15:13:32.616885 8160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:32.633469 8160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:32.636798 8160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:32.637046 8160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:32.637710 8160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:32.638604 8160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:32.638706 8160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:32.638795 8160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:32.638844 8160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:32.648713 8160 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:46649
I20260807 15:13:32.648768 8354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:46649 every 8 connection(s)
I20260807 15:13:32.649850 8160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:13:32.650941 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 8160
I20260807 15:13:32.654376 8359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:32.652473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.67:49775)
0807 15:13:32.653034 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:32.653164 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":132,"threads_started":1}
W20260807 15:13:32.654963 8355 heartbeater.cc:644] Failed to heartbeat to 127.15.226.126:36145 (0 consecutive failures): Network error: Failed to ping master at 127.15.226.126:36145: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
I20260807 15:13:32.655668 8359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:32.655423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.67:50043)
0807 15:13:32.655555 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:32.655608 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 15:13:32.656528 8359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:32.656289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.67:54815)
0807 15:13:32.656424 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:32.656474 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 15:13:32.656850 8355 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 15:13:33.328819 8362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:33.328243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.65:53029)
0807 15:13:33.328629 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:33.328704 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":97,"threads_started":1}
I20260807 15:13:33.452632 8364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:33.452163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.66:44801)
0807 15:13:33.452454 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:33.452521 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":96,"threads_started":1}
I20260807 15:13:33.658123 8365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:33.657595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.67:43409)
0807 15:13:33.657918 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:33.658000 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":84,"threads_started":1}
I20260807 15:13:33.941450 8367 raft_consensus.cc:482] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:13:33.941754 8367 raft_consensus.cc:504] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } } peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } }
I20260807 15:13:33.943101 8367 leader_election.cc:288] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 489972c8b9ef4a339aa8c09de646d108 (127.15.226.67:46649), caee2f5650aa4f1e8ae25fc770229752 (127.15.226.66:43769)
Aug 07 15:13:33 dist-test-slave-3kk6 krb5kdc[6008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115612, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.66@KRBTEST.COM
Aug 07 15:13:33 dist-test-slave-3kk6 krb5kdc[6008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115612, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.67@KRBTEST.COM
I20260807 15:13:33.958349 8372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:33.943809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:43375 (local address 127.15.226.67:46649)
0807 15:13:33.944163 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:33.944186 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:33.944223 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:13:33.944930 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:33.944936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:33.945285 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:33.945610 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:33.947117 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:33.948112 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:33.949370 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:33.949859 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:33.950047 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:33.953495 (+ 3448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:33.953501 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:33.953512 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:33.953545 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:33.955593 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:33.956160 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:33.956164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:33.956169 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:33.956293 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:33.956651 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:33.956654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:33.956656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:33.957116 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:33.957265 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:33.957842 (+ 577us) server_negotiation.cc:299] Negotiation successful
0807 15:13:33.958158 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":121,"threads_started":1}
I20260807 15:13:33.958335 8368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:33.943388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:43769 (local address 127.15.226.65:42571)
0807 15:13:33.943885 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:33.943901 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:33.944014 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:33.945694 (+ 1680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:33.945697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:33.946276 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:33.946879 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:33.946903 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:33.948274 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:33.948278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:33.949254 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:33.949265 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:33.949999 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:33.950002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:33.950241 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:33.951033 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:13:33.951050 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:33.952962 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:33.955545 (+ 2583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:33.955552 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:33.955568 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:33.955997 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:33.956391 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:33.956395 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:33.956398 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:33.956559 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:33.957243 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:33.957250 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:33.957524 (+ 274us) client_negotiation.cc:769] Sending connection context
0807 15:13:33.957727 (+ 203us) client_negotiation.cc:240] Negotiation successful
0807 15:13:33.958133 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":203,"spinlock_wait_cycles":35840,"thread_start_us":54,"threads_started":1}
I20260807 15:13:33.958851 8370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:33.943388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:46649 (local address 127.15.226.65:43375)
0807 15:13:33.943739 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:33.943756 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:33.943889 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:33.945673 (+ 1784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:33.945680 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:33.946286 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:33.946951 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:33.946965 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:33.948272 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:33.948275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:33.949232 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:33.949243 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:33.949999 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:33.950002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:33.950232 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:33.951094 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:13:33.951106 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:33.953268 (+ 2162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:33.955731 (+ 2463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:33.955734 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:33.955736 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:33.956010 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:33.956429 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:33.956432 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:33.956433 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:33.956547 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:33.957242 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:33.957249 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:33.957525 (+ 276us) client_negotiation.cc:769] Sending connection context
0807 15:13:33.957726 (+ 201us) client_negotiation.cc:240] Negotiation successful
0807 15:13:33.958260 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":301,"spinlock_wait_cycles":8704,"thread_start_us":93,"threads_started":1}
I20260807 15:13:33.959165 8371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:33.943597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:42571 (local address 127.15.226.66:43769)
0807 15:13:33.944162 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:33.944186 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:33.944223 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:13:33.944926 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:33.944931 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:33.945285 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:33.945609 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:33.947093 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:33.948112 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:33.949388 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:33.949859 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:33.950046 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:33.953258 (+ 3212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:33.953265 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:33.953284 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:33.953323 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:33.955387 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:33.956127 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:33.956135 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:33.956145 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:33.956276 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:33.956679 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:33.956682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:33.956683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:33.957115 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:33.957265 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:33.957846 (+ 581us) server_negotiation.cc:299] Negotiation successful
0807 15:13:33.958262 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":182,"threads_started":1}
W20260807 15:13:33.961519 7802 leader_election.cc:341] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 489972c8b9ef4a339aa8c09de646d108 (127.15.226.67:46649): Not found: Tablet not found: fb92990b3a534ba09079d73b35136d19
W20260807 15:13:33.961706 7808 leader_election.cc:341] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer caee2f5650aa4f1e8ae25fc770229752 (127.15.226.66:43769): Not found: Tablet not found: fb92990b3a534ba09079d73b35136d19
I20260807 15:13:33.961794 7808 leader_election.cc:302] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f1e4a721b8f4c6e8ff8325b0e48db50; no voters: 489972c8b9ef4a339aa8c09de646d108, caee2f5650aa4f1e8ae25fc770229752
I20260807 15:13:33.962062 8367 raft_consensus.cc:2726] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:13:34.330230 8370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:34.329876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.65:42327)
0807 15:13:34.330054 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:34.330136 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 15:13:34.454315 8373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:34.453653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.66:55169)
0807 15:13:34.454056 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:34.454156 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":110,"threads_started":1}
I20260807 15:13:34.659585 8374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:34.659091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.67:36689)
0807 15:13:34.659411 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:34.659477 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":114,"threads_started":1}
I20260807 15:13:35.331743 8376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:35.331208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.65:43695)
0807 15:13:35.331543 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:35.331619 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":95,"threads_started":1}
I20260807 15:13:35.455938 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:35.455448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.66:42355)
0807 15:13:35.455741 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:35.455831 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":92,"threads_started":1}
I20260807 15:13:35.606151 8378 raft_consensus.cc:482] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:13:35.606307 8378 raft_consensus.cc:504] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } } peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } }
I20260807 15:13:35.606932 8378 leader_election.cc:288] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 489972c8b9ef4a339aa8c09de646d108 (127.15.226.67:46649), caee2f5650aa4f1e8ae25fc770229752 (127.15.226.66:43769)
W20260807 15:13:35.607878 7808 leader_election.cc:341] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer caee2f5650aa4f1e8ae25fc770229752 (127.15.226.66:43769): Not found: Tablet not found: fb92990b3a534ba09079d73b35136d19
W20260807 15:13:35.608011 7802 leader_election.cc:341] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 489972c8b9ef4a339aa8c09de646d108 (127.15.226.67:46649): Not found: Tablet not found: fb92990b3a534ba09079d73b35136d19
I20260807 15:13:35.608181 7802 leader_election.cc:302] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f1e4a721b8f4c6e8ff8325b0e48db50; no voters: 489972c8b9ef4a339aa8c09de646d108, caee2f5650aa4f1e8ae25fc770229752
I20260807 15:13:35.608412 8378 raft_consensus.cc:2726] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:13:35.661175 8379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:35.660640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.67:54221)
0807 15:13:35.660990 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:35.661068 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":103,"threads_started":1}
I20260807 15:13:36.333247 8382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:36.332750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.65:45913)
0807 15:13:36.333059 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:36.333139 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":92,"threads_started":1}
I20260807 15:13:36.457458 8384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:36.456944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.66:51909)
0807 15:13:36.457278 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:36.457353 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":98,"threads_started":1}
I20260807 15:13:36.662992 8385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:36.662327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.67:33059)
0807 15:13:36.662778 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:36.662868 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":100,"threads_started":1}
I20260807 15:13:37.335016 8387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:37.334427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.65:44439)
0807 15:13:37.334837 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:37.334913 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":88,"threads_started":1}
I20260807 15:13:37.355886 8389 raft_consensus.cc:482] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:13:37.356050 8389 raft_consensus.cc:504] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } } peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } }
I20260807 15:13:37.356568 8389 leader_election.cc:288] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 489972c8b9ef4a339aa8c09de646d108 (127.15.226.67:46649), caee2f5650aa4f1e8ae25fc770229752 (127.15.226.66:43769)
W20260807 15:13:37.357467 7802 leader_election.cc:341] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 489972c8b9ef4a339aa8c09de646d108 (127.15.226.67:46649): Not found: Tablet not found: fb92990b3a534ba09079d73b35136d19
W20260807 15:13:37.357630 7808 leader_election.cc:341] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer caee2f5650aa4f1e8ae25fc770229752 (127.15.226.66:43769): Not found: Tablet not found: fb92990b3a534ba09079d73b35136d19
I20260807 15:13:37.357728 7808 leader_election.cc:302] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f1e4a721b8f4c6e8ff8325b0e48db50; no voters: 489972c8b9ef4a339aa8c09de646d108, caee2f5650aa4f1e8ae25fc770229752
I20260807 15:13:37.357931 8389 raft_consensus.cc:2726] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:13:37.459115 8390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:37.458478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.66:35051)
0807 15:13:37.458906 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:37.459009 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":133,"threads_started":1}
I20260807 15:13:37.664875 8391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:37.664207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.67:39337)
0807 15:13:37.664616 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:37.664726 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":107,"threads_started":1}
I20260807 15:13:38.336547 8394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:38.336010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.65:59765)
0807 15:13:38.336353 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:38.336438 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":128,"threads_started":1}
I20260807 15:13:38.460758 8395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:38.460218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.66:41711)
0807 15:13:38.460570 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:38.460650 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":96,"threads_started":1}
I20260807 15:13:38.666358 8399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:38.665841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.67:54185)
0807 15:13:38.666193 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:38.666254 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":121,"threads_started":1}
I20260807 15:13:39.338394 8401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:39.337684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.65:34511)
0807 15:13:39.338081 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:39.338171 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":107,"threads_started":1}
I20260807 15:13:39.462483 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:39.461894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.66:35879)
0807 15:13:39.462301 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:39.462376 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":97,"threads_started":1}
I20260807 15:13:39.668187 8404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:39.667498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.67:57025)
0807 15:13:39.667942 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:39.668016 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":93,"threads_started":1}
I20260807 15:13:40.340121 8405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:40.339555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.65:43959)
0807 15:13:40.339928 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:40.340008 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":92,"threads_started":1}
I20260807 15:13:40.464263 8408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:40.463628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.66:52859)
0807 15:13:40.464053 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:40.464139 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":149,"threads_started":1}
I20260807 15:13:40.669966 8410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:40.669276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.67:56885)
0807 15:13:40.669713 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:40.669833 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":151,"threads_started":1}
I20260807 15:13:41.341845 8412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:41.341300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.65:34273)
0807 15:13:41.341645 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:41.341732 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":137,"threads_started":1}
I20260807 15:13:41.465984 8413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:41.465425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.66:59029)
0807 15:13:41.465787 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:41.465865 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":112,"threads_started":1}
I20260807 15:13:41.671861 8415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:41.671268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.67:46311)
0807 15:13:41.671647 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:41.671747 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":133,"threads_started":1}
I20260807 15:13:42.189949 8417 raft_consensus.cc:482] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:13:42.190147 8417 raft_consensus.cc:504] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489972c8b9ef4a339aa8c09de646d108" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46649 } } peers { permanent_uuid: "caee2f5650aa4f1e8ae25fc770229752" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43769 } } peers { permanent_uuid: "3f1e4a721b8f4c6e8ff8325b0e48db50" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 36265 } }
I20260807 15:13:42.190735 8417 leader_election.cc:288] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 489972c8b9ef4a339aa8c09de646d108 (127.15.226.67:46649), caee2f5650aa4f1e8ae25fc770229752 (127.15.226.66:43769)
W20260807 15:13:42.191661 7802 leader_election.cc:341] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 489972c8b9ef4a339aa8c09de646d108 (127.15.226.67:46649): Not found: Tablet not found: fb92990b3a534ba09079d73b35136d19
W20260807 15:13:42.191825 7808 leader_election.cc:341] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer caee2f5650aa4f1e8ae25fc770229752 (127.15.226.66:43769): Not found: Tablet not found: fb92990b3a534ba09079d73b35136d19
I20260807 15:13:42.191897 7808 leader_election.cc:302] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3f1e4a721b8f4c6e8ff8325b0e48db50; no voters: 489972c8b9ef4a339aa8c09de646d108, caee2f5650aa4f1e8ae25fc770229752
I20260807 15:13:42.192055 8417 raft_consensus.cc:2726] T fb92990b3a534ba09079d73b35136d19 P 3f1e4a721b8f4c6e8ff8325b0e48db50 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 15:13:42.343590 8418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:42.343071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.65:34539)
0807 15:13:42.343406 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:42.343481 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":118,"threads_started":1}
I20260807 15:13:42.467679 8421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:42.467128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.66:35965)
0807 15:13:42.467479 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:42.467559 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":95,"threads_started":1}
I20260807 15:13:42.673774 8424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:42.673006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.67:56453)
0807 15:13:42.673573 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:42.673655 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":120,"threads_started":1}
I20260807 15:13:43.345057 8425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:43.344556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.65:53037)
0807 15:13:43.344884 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:43.344952 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":114,"threads_started":1}
I20260807 15:13:43.469059 8427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:43.468528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.66:60441)
0807 15:13:43.468871 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:43.468944 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":92,"threads_started":1}
I20260807 15:13:43.675452 8428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:43.674914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.67:37609)
0807 15:13:43.675276 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:43.675345 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":116,"threads_started":1}
I20260807 15:13:44.346428 8429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:44.345970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.65:40813)
0807 15:13:44.346251 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:44.346321 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":89,"threads_started":1}
I20260807 15:13:44.470746 8431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:44.470182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.66:56717)
0807 15:13:44.470510 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:44.470580 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":89,"threads_started":1}
I20260807 15:13:44.676994 8432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:44.676480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.67:32795)
0807 15:13:44.676808 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:44.676878 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":116,"threads_started":1}
I20260807 15:13:45.347937 8433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:45.347452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.65:48433)
0807 15:13:45.347758 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:45.347833 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":97,"threads_started":1}
I20260807 15:13:45.472124 8435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:45.471648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.66:37781)
0807 15:13:45.471963 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:45.472022 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":97,"threads_started":1}
I20260807 15:13:45.678479 8437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:45.677965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.67:52707)
0807 15:13:45.678276 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:45.678345 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":96,"threads_started":1}
I20260807 15:13:46.349503 8438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:46.348952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.65:57243)
0807 15:13:46.349314 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:46.349399 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":103,"threads_started":1}
I20260807 15:13:46.473635 8439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:46.473005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.66:35659)
0807 15:13:46.473452 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:46.473530 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":103,"threads_started":1}
I20260807 15:13:46.679919 8440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:46.679428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.67:48411)
0807 15:13:46.679735 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:46.679812 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":114,"threads_started":1}
I20260807 15:13:47.351229 8443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:47.350726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.65:51129)
0807 15:13:47.351028 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:47.351091 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":91,"threads_started":1}
I20260807 15:13:47.475147 8444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:47.474603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.66:38649)
0807 15:13:47.474935 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:47.475041 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":134,"threads_started":1}
I20260807 15:13:47.662843 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 7716
I20260807 15:13:47.681623 8445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:47.681121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36145 (local address 127.15.226.67:54183)
0807 15:13:47.681453 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:47.681520 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.126:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":111,"threads_started":1}
I20260807 15:13:47.683662 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 7974
I20260807 15:13:47.698987 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 8160
2026-08-07T15:13:47Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59862 ms)
[----------] 30 tests from SecurityITest (262560 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:13:47 dist-test-slave-3kk6 krb5kdc[8450](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:13:47 dist-test-slave-3kk6 krb5kdc[8450](info): set up 2 sockets
Aug 07 15:13:47 dist-test-slave-3kk6 krb5kdc[8450](info): commencing operation
krb5kdc: starting...
W20260807 15:13:48.996042 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:13:49 dist-test-slave-3kk6 krb5kdc[8450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:13:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:13:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:13:49.135516 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:34585
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:39319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:34585
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:13:49.254806 8473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:49.255167 8473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:49.255249 8473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:49.259508 8473 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:13:49.259617 8473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:49.259654 8473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:49.259694 8473 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:13:49.259737 8473 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:13:49.265408 8473 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:39319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:34585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:34585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:49.266605 8473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:49.267753 8473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:49.276999 8485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:49.277081 8488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:49.276998 8484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:49.277165 8473 server_base.cc:1034] running on GCE node
I20260807 15:13:49.278044 8473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:49.279400 8473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:49.280580 8473 hybrid_clock.cc:648] HybridClock initialized: now 1786115629280549 us; error 52 us; skew 500 ppm
Aug 07 15:13:49 dist-test-slave-3kk6 krb5kdc[8450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115629, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:49.284580 8473 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:13:49.285756 8473 webserver.cc:460] Webserver started at http://127.15.226.126:45993/ using document root <none> and password file <none>
I20260807 15:13:49.286429 8473 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:49.286538 8473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:49.286857 8473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:13:49.289273 8473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "e8040d87defd4f6696094d9f5ffcdea4"
format_stamp: "Formatted at 2026-08-07 15:13:49 on dist-test-slave-3kk6"
server_key: "afb408d63d8f9fc65c37c55f4ee4cca3"
server_key_iv: "7c096d0aeb04f0510d652414192317ff"
server_key_version: "encryptionkey@0"
I20260807 15:13:49.289961 8473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "e8040d87defd4f6696094d9f5ffcdea4"
format_stamp: "Formatted at 2026-08-07 15:13:49 on dist-test-slave-3kk6"
server_key: "afb408d63d8f9fc65c37c55f4ee4cca3"
server_key_iv: "7c096d0aeb04f0510d652414192317ff"
server_key_version: "encryptionkey@0"
I20260807 15:13:49.294749 8473 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260807 15:13:49.297610 8496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:49.299260 8473 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:13:49.299459 8473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "e8040d87defd4f6696094d9f5ffcdea4"
format_stamp: "Formatted at 2026-08-07 15:13:49 on dist-test-slave-3kk6"
server_key: "afb408d63d8f9fc65c37c55f4ee4cca3"
server_key_iv: "7c096d0aeb04f0510d652414192317ff"
server_key_version: "encryptionkey@0"
I20260807 15:13:49.299623 8473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:49.316344 8473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:49.320354 8473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:49.320639 8473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:49.329315 8473 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:34585
I20260807 15:13:49.329375 8573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:34585 every 8 connection(s)
I20260807 15:13:49.330476 8473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:13:49.333690 8575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:49.334163 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 8473
I20260807 15:13:49.334329 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:13:49.334745 16265 external_mini_cluster.cc:1428] Setting key 859e22fc17a5b5ec761def7564cee689
I20260807 15:13:49.340840 8575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8040d87defd4f6696094d9f5ffcdea4: Bootstrap starting.
I20260807 15:13:49.343863 8575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8040d87defd4f6696094d9f5ffcdea4: Neither blocks nor log segments found. Creating new log.
Aug 07 15:13:49 dist-test-slave-3kk6 krb5kdc[8450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:13:49.344795 8575 log.cc:824] T 00000000000000000000000000000000 P e8040d87defd4f6696094d9f5ffcdea4: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:49.347023 8575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8040d87defd4f6696094d9f5ffcdea4: No bootstrap required, opened a new log
I20260807 15:13:49.350785 8580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:49.336314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36734 (local address 127.15.226.126:34585)
0807 15:13:49.336805 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:49.336821 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:49.336905 (+ 84us) server_negotiation.cc:407] Connection header received
0807 15:13:49.337605 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:49.337623 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:49.337995 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:49.338290 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:49.339380 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:49.340418 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:49.341638 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:49.342186 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:49.342326 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:49.345457 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:49.345462 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:49.345478 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:49.345508 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:49.347497 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:49.348020 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:49.348025 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:49.348030 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:49.348152 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:49.348475 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:49.348479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:49.348482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:49.348907 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:49.349139 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:49.349468 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 15:13:49.349862 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":157,"threads_started":1}
I20260807 15:13:49.350955 8575 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8040d87defd4f6696094d9f5ffcdea4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8040d87defd4f6696094d9f5ffcdea4" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34585 } }
I20260807 15:13:49.351274 8575 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8040d87defd4f6696094d9f5ffcdea4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:49.351356 8575 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8040d87defd4f6696094d9f5ffcdea4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8040d87defd4f6696094d9f5ffcdea4, State: Initialized, Role: FOLLOWER
I20260807 15:13:49.351898 8575 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8040d87defd4f6696094d9f5ffcdea4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8040d87defd4f6696094d9f5ffcdea4" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34585 } }
I20260807 15:13:49.352104 8575 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8040d87defd4f6696094d9f5ffcdea4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:13:49.352177 8575 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8040d87defd4f6696094d9f5ffcdea4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:13:49.352288 8575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8040d87defd4f6696094d9f5ffcdea4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:49.353664 8575 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8040d87defd4f6696094d9f5ffcdea4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8040d87defd4f6696094d9f5ffcdea4" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34585 } }
I20260807 15:13:49.353979 8575 leader_election.cc:302] T 00000000000000000000000000000000 P e8040d87defd4f6696094d9f5ffcdea4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8040d87defd4f6696094d9f5ffcdea4; no voters:
I20260807 15:13:49.354288 8575 leader_election.cc:288] T 00000000000000000000000000000000 P e8040d87defd4f6696094d9f5ffcdea4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:13:49.354468 8586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8040d87defd4f6696094d9f5ffcdea4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:13:49.354773 8586 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8040d87defd4f6696094d9f5ffcdea4 [term 1 LEADER]: Becoming Leader. State: Replica: e8040d87defd4f6696094d9f5ffcdea4, State: Running, Role: LEADER
I20260807 15:13:49.355149 8586 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8040d87defd4f6696094d9f5ffcdea4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8040d87defd4f6696094d9f5ffcdea4" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34585 } }
I20260807 15:13:49.355712 8575 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8040d87defd4f6696094d9f5ffcdea4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:13:49.357084 8589 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8040d87defd4f6696094d9f5ffcdea4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8040d87defd4f6696094d9f5ffcdea4. Latest consensus state: current_term: 1 leader_uuid: "e8040d87defd4f6696094d9f5ffcdea4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8040d87defd4f6696094d9f5ffcdea4" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34585 } } }
I20260807 15:13:49.357084 8588 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8040d87defd4f6696094d9f5ffcdea4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8040d87defd4f6696094d9f5ffcdea4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8040d87defd4f6696094d9f5ffcdea4" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 34585 } } }
I20260807 15:13:49.357251 8589 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8040d87defd4f6696094d9f5ffcdea4 [sys.catalog]: This master's current role is: LEADER
I20260807 15:13:49.357251 8588 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8040d87defd4f6696094d9f5ffcdea4 [sys.catalog]: This master's current role is: LEADER
I20260807 15:13:49.357718 8600 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:13:49.361224 8600 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:13:49.367933 8600 catalog_manager.cc:1347] Generated new cluster ID: c2137b71e1c8431092586d0291d9b1cc
I20260807 15:13:49.368037 8600 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:13:49.387151 8600 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:13:49.388235 8600 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:13:49.396278 8600 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8040d87defd4f6696094d9f5ffcdea4: Generated new TSK 0
I20260807 15:13:49.397066 8600 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.226.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
I20260807 15:13:49.438330 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34585
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:39319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:13:49.563176 8620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:49.563501 8620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:49.563549 8620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:49.567831 8620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:49.567915 8620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:49.568014 8620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:13:49.573900 8620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:39319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:49.575192 8620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:49.576392 8620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:49.587957 8629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:49.587999 8628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:49.587984 8620 server_base.cc:1034] running on GCE node
W20260807 15:13:49.588018 8632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:49.588662 8620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:49.589452 8620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:49.590632 8620 hybrid_clock.cc:648] HybridClock initialized: now 1786115629590594 us; error 56 us; skew 500 ppm
Aug 07 15:13:49 dist-test-slave-3kk6 krb5kdc[8450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115629, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:49.593904 8620 init.cc:377] Logged in from keytab as kudu/127.15.226.65@KRBTEST.COM (short username kudu)
I20260807 15:13:49.594949 8620 webserver.cc:460] Webserver started at http://127.15.226.65:41921/ using document root <none> and password file <none>
I20260807 15:13:49.595530 8620 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:49.595603 8620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:49.595793 8620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:13:49.598016 8620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "d05c9dac1c284ac8a85aacbc92a96e06"
format_stamp: "Formatted at 2026-08-07 15:13:49 on dist-test-slave-3kk6"
server_key: "8aa46128eaf57144bfc83ed3d6b8a312"
server_key_iv: "3e5d2079cd73c9393972a0acdcc5b2e5"
server_key_version: "encryptionkey@0"
I20260807 15:13:49.598589 8620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "d05c9dac1c284ac8a85aacbc92a96e06"
format_stamp: "Formatted at 2026-08-07 15:13:49 on dist-test-slave-3kk6"
server_key: "8aa46128eaf57144bfc83ed3d6b8a312"
server_key_iv: "3e5d2079cd73c9393972a0acdcc5b2e5"
server_key_version: "encryptionkey@0"
I20260807 15:13:49.602878 8620 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:13:49.605794 8641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:49.607450 8620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:13:49.607607 8620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "d05c9dac1c284ac8a85aacbc92a96e06"
format_stamp: "Formatted at 2026-08-07 15:13:49 on dist-test-slave-3kk6"
server_key: "8aa46128eaf57144bfc83ed3d6b8a312"
server_key_iv: "3e5d2079cd73c9393972a0acdcc5b2e5"
server_key_version: "encryptionkey@0"
I20260807 15:13:49.607743 8620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:49.622892 8620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:49.626861 8620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:49.627111 8620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:49.627794 8620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:49.628712 8620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:49.628795 8620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:49.628871 8620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:49.628916 8620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:49.641011 8620 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:38205
I20260807 15:13:49.641063 8814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:38205 every 8 connection(s)
I20260807 15:13:49.642045 8620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:13:49.647511 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 8620
I20260807 15:13:49.647684 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:13:49.648133 16265 external_mini_cluster.cc:1428] Setting key a08e4b02c0df5b6e95e214f9fc928938
Aug 07 15:13:49 dist-test-slave-3kk6 krb5kdc[8450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115629, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:13:49.659801 8580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:49.644300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:50893 (local address 127.15.226.126:34585)
0807 15:13:49.644479 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:49.644483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:49.645482 (+ 999us) server_negotiation.cc:407] Connection header received
0807 15:13:49.647312 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:49.647316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:49.647378 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:49.647465 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:49.649166 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:49.649927 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:49.651888 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:49.652138 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:49.652226 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:49.655319 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:49.655323 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:49.655325 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:49.655374 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:49.657181 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:49.657895 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:49.657898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:49.657899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:49.657984 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:49.658431 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:49.658435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:49.658437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:49.658731 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:49.658854 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:49.659449 (+ 595us) server_negotiation.cc:299] Negotiation successful
0807 15:13:49.659606 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
WARNING: no policy specified for kudu/127.15.226.66@KRBTEST.COM; defaulting to no policy
I20260807 15:13:49.660972 8819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:49.644700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34585 (local address 127.15.226.65:50893)
0807 15:13:49.645283 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:49.645344 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:49.646995 (+ 1651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:49.647728 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:49.647742 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:49.648346 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:49.649009 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:49.649026 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:49.650109 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:49.650114 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:49.651727 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:49.651745 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:49.652329 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:49.652335 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:49.652600 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:49.653345 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:13:49.653363 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:49.655093 (+ 1730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:49.657323 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:49.657338 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:49.657358 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:49.657760 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:49.658090 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:49.658093 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:49.658095 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:49.658279 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:49.658848 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:49.658857 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:49.659172 (+ 315us) client_negotiation.cc:769] Sending connection context
0807 15:13:49.659393 (+ 221us) client_negotiation.cc:240] Negotiation successful
0807 15:13:49.659791 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":170,"threads_started":1}
I20260807 15:13:49.662599 8815 heartbeater.cc:347] Connected to a master server at 127.15.226.126:34585
I20260807 15:13:49.662959 8815 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.66@KRBTEST.COM" created.
I20260807 15:13:49.663674 8815 heartbeater.cc:511] Master 127.15.226.126:34585 requested a full tablet report, sending...
I20260807 15:13:49.665637 8518 ts_manager.cc:194] Registered new tserver with Master: d05c9dac1c284ac8a85aacbc92a96e06 (127.15.226.65:38205)
I20260807 15:13:49.666990 8518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.65@KRBTEST.COM'} at 127.15.226.65:50893
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
I20260807 15:13:49.682015 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34585
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:39319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:13:49.807761 8823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:49.808218 8823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:49.808322 8823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:49.812768 8823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:49.812878 8823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:49.812973 8823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:13:49.818872 8823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:39319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:49.820137 8823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:49.821280 8823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:49.832105 8833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:49.832123 8830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:49.832157 8823 server_base.cc:1034] running on GCE node
W20260807 15:13:49.832099 8829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:49.832852 8823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:49.833622 8823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:49.834802 8823 hybrid_clock.cc:648] HybridClock initialized: now 1786115629834794 us; error 34 us; skew 500 ppm
Aug 07 15:13:49 dist-test-slave-3kk6 krb5kdc[8450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115629, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:49.838033 8823 init.cc:377] Logged in from keytab as kudu/127.15.226.66@KRBTEST.COM (short username kudu)
I20260807 15:13:49.839082 8823 webserver.cc:460] Webserver started at http://127.15.226.66:45243/ using document root <none> and password file <none>
I20260807 15:13:49.839658 8823 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:49.839720 8823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:49.839974 8823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:13:49.842191 8823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "1d095da5fe0148aab17291957cf571c1"
format_stamp: "Formatted at 2026-08-07 15:13:49 on dist-test-slave-3kk6"
server_key: "5b29d3a16cc16796989585fad779dd98"
server_key_iv: "613c6f35ae28bf8117f7bf88d7e2e318"
server_key_version: "encryptionkey@0"
I20260807 15:13:49.842882 8823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "1d095da5fe0148aab17291957cf571c1"
format_stamp: "Formatted at 2026-08-07 15:13:49 on dist-test-slave-3kk6"
server_key: "5b29d3a16cc16796989585fad779dd98"
server_key_iv: "613c6f35ae28bf8117f7bf88d7e2e318"
server_key_version: "encryptionkey@0"
I20260807 15:13:49.847090 8823 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:13:49.849741 8842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:49.851298 8823 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:13:49.851411 8823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "1d095da5fe0148aab17291957cf571c1"
format_stamp: "Formatted at 2026-08-07 15:13:49 on dist-test-slave-3kk6"
server_key: "5b29d3a16cc16796989585fad779dd98"
server_key_iv: "613c6f35ae28bf8117f7bf88d7e2e318"
server_key_version: "encryptionkey@0"
I20260807 15:13:49.851526 8823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:49.871138 8823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:49.875098 8823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:49.875277 8823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:49.875892 8823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:49.876753 8823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:49.876806 8823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:49.876848 8823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:49.876870 8823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:49.887161 8823 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:44453
I20260807 15:13:49.887187 9014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:44453 every 8 connection(s)
I20260807 15:13:49.888233 8823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:13:49.890976 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 8823
I20260807 15:13:49.891114 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:13:49.891438 16265 external_mini_cluster.cc:1428] Setting key 7103f98b46eb4dbcb2bfafd0fd53f7b2
Aug 07 15:13:49 dist-test-slave-3kk6 krb5kdc[8450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115629, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.226.67@KRBTEST.COM; defaulting to no policy
I20260807 15:13:49.905862 8580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:49.890335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:39807 (local address 127.15.226.126:34585)
0807 15:13:49.890497 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:49.890501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:49.891526 (+ 1025us) server_negotiation.cc:407] Connection header received
0807 15:13:49.893244 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:49.893248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:49.893306 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:49.893399 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:49.894686 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:49.895583 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:49.897464 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:49.897692 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:49.897810 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:49.901351 (+ 3541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:49.901355 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:49.901357 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:49.901387 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:49.903130 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:49.903952 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:49.903955 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:49.903957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:49.904047 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:49.904419 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:49.904421 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:49.904422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:49.904735 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:49.904869 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:49.905539 (+ 670us) server_negotiation.cc:299] Negotiation successful
0807 15:13:49.905686 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.67@KRBTEST.COM" created.
I20260807 15:13:49.907089 9017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:49.890806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34585 (local address 127.15.226.66:39807)
0807 15:13:49.891335 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:49.891398 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:49.893009 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:49.893574 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:49.893581 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:49.893961 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:49.894494 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:49.894509 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:49.895732 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:49.895738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:49.897307 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:49.897322 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:49.897879 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:49.897886 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:49.898135 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:49.898945 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:13:49.898972 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:49.901086 (+ 2114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:49.903292 (+ 2206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:49.903299 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:49.903320 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:49.903786 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:49.904158 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:49.904161 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:49.904162 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:49.904326 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:49.904869 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:49.904878 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:49.905261 (+ 383us) client_negotiation.cc:769] Sending connection context
0807 15:13:49.905492 (+ 231us) client_negotiation.cc:240] Negotiation successful
0807 15:13:49.905951 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":143,"threads_started":1}
I20260807 15:13:49.910338 9015 heartbeater.cc:347] Connected to a master server at 127.15.226.126:34585
I20260807 15:13:49.910640 9015 heartbeater.cc:464] Registering TS with master...
I20260807 15:13:49.911289 9015 heartbeater.cc:511] Master 127.15.226.126:34585 requested a full tablet report, sending...
I20260807 15:13:49.912529 8518 ts_manager.cc:194] Registered new tserver with Master: 1d095da5fe0148aab17291957cf571c1 (127.15.226.66:44453)
I20260807 15:13:49.913206 8518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.66@KRBTEST.COM'} at 127.15.226.66:39807
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
I20260807 15:13:49.926083 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34585
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:39319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:13:50.050369 9022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:50.050743 9022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:50.050798 9022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:50.055066 9022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:50.055135 9022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:50.055243 9022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:13:50.061041 9022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:39319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:34585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:50.062141 9022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:50.063263 9022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:50.073769 9030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:50.073832 9029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:50.073872 9033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:50.074426 9022 server_base.cc:1034] running on GCE node
I20260807 15:13:50.074945 9022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:50.075572 9022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:50.076769 9022 hybrid_clock.cc:648] HybridClock initialized: now 1786115630076740 us; error 60 us; skew 500 ppm
Aug 07 15:13:50 dist-test-slave-3kk6 krb5kdc[8450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115630, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:50.080150 9022 init.cc:377] Logged in from keytab as kudu/127.15.226.67@KRBTEST.COM (short username kudu)
I20260807 15:13:50.081174 9022 webserver.cc:460] Webserver started at http://127.15.226.67:39631/ using document root <none> and password file <none>
I20260807 15:13:50.081786 9022 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:50.081915 9022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:50.082175 9022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:13:50.084558 9022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "143615220c774a93834c0844149b7a8d"
format_stamp: "Formatted at 2026-08-07 15:13:50 on dist-test-slave-3kk6"
server_key: "623319a454585eec3df46db8be373d08"
server_key_iv: "84dd1bf42832beb0cf7e0ecb52759e0d"
server_key_version: "encryptionkey@0"
I20260807 15:13:50.085197 9022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "143615220c774a93834c0844149b7a8d"
format_stamp: "Formatted at 2026-08-07 15:13:50 on dist-test-slave-3kk6"
server_key: "623319a454585eec3df46db8be373d08"
server_key_iv: "84dd1bf42832beb0cf7e0ecb52759e0d"
server_key_version: "encryptionkey@0"
I20260807 15:13:50.089471 9022 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260807 15:13:50.092370 9045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:50.093684 9022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:13:50.093878 9022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "143615220c774a93834c0844149b7a8d"
format_stamp: "Formatted at 2026-08-07 15:13:50 on dist-test-slave-3kk6"
server_key: "623319a454585eec3df46db8be373d08"
server_key_iv: "84dd1bf42832beb0cf7e0ecb52759e0d"
server_key_version: "encryptionkey@0"
I20260807 15:13:50.094017 9022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:50.110442 9022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:50.114193 9022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:50.114427 9022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:50.115151 9022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:50.116078 9022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:50.116165 9022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:50.116257 9022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:50.116309 9022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:50.126307 9022 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:38251
I20260807 15:13:50.126343 9234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:38251 every 8 connection(s)
I20260807 15:13:50.127480 9022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:13:50.135090 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 9022
I20260807 15:13:50.135249 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:13:50.135617 16265 external_mini_cluster.cc:1428] Setting key 4819338e7e7274c617de4792941d1722
Aug 07 15:13:50 dist-test-slave-3kk6 krb5kdc[8450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115630, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:13:50.143082 8580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:50.129434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:52447 (local address 127.15.226.126:34585)
0807 15:13:50.129596 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:50.129601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:50.130500 (+ 899us) server_negotiation.cc:407] Connection header received
0807 15:13:50.132019 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:50.132023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:50.132087 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:50.132191 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:50.133431 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.133996 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.135608 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.135973 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.136109 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:50.139305 (+ 3196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:50.139309 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:50.139312 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:50.139353 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:50.140730 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:50.141377 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:50.141385 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:50.141387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:50.141471 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:50.141862 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:50.141865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:50.141867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:50.142075 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:50.142190 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:50.142765 (+ 575us) server_negotiation.cc:299] Negotiation successful
0807 15:13:50.142914 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 15:13:50.143985 9238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:50.129806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:34585 (local address 127.15.226.67:52447)
0807 15:13:50.130315 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:50.130374 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:50.131782 (+ 1408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:50.132386 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:50.132394 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:50.132765 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:50.133267 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:50.133281 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.134136 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.134142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:50.135460 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:50.135476 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.136149 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.136173 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:50.136398 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:50.137193 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:13:50.137220 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:50.139127 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:50.140933 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:50.140940 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:50.140962 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:50.141246 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:50.141579 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:50.141582 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:50.141584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:50.141744 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:50.142189 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:50.142198 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:50.142487 (+ 289us) client_negotiation.cc:769] Sending connection context
0807 15:13:50.142716 (+ 229us) client_negotiation.cc:240] Negotiation successful
0807 15:13:50.143098 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":124,"threads_started":1}
I20260807 15:13:50.145172 9236 heartbeater.cc:347] Connected to a master server at 127.15.226.126:34585
I20260807 15:13:50.145460 9236 heartbeater.cc:464] Registering TS with master...
I20260807 15:13:50.146018 9236 heartbeater.cc:511] Master 127.15.226.126:34585 requested a full tablet report, sending...
I20260807 15:13:50.147179 8518 ts_manager.cc:194] Registered new tserver with Master: 143615220c774a93834c0844149b7a8d (127.15.226.67:38251)
I20260807 15:13:50.147785 8518 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.67@KRBTEST.COM'} at 127.15.226.67:52447
I20260807 15:13:50.150081 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:13:50.161163 8580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:50.152360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36742 (local address 127.15.226.126:34585)
0807 15:13:50.152497 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:50.152501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:50.152597 (+ 96us) server_negotiation.cc:407] Connection header received
0807 15:13:50.152703 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:50.152705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:50.152744 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:50.152834 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:50.153605 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.154140 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.155105 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.155430 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.155524 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:50.157433 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:50.157437 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:50.157439 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:50.157465 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:50.159067 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:50.159562 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:50.159567 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:50.159570 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:50.159663 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:50.159990 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:50.159995 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:50.159997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:50.160257 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:50.160493 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:50.160778 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 15:13:50.160975 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 15:13:50.164534 8518 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36742:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:13:50.167764 8518 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:13:50.183216 9258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:50.177611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:44453 (local address 127.0.0.1:53974)
0807 15:13:50.178467 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:50.178479 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:50.178601 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:50.179020 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:50.179023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:50.179041 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:50.179138 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:50.179144 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.180105 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.180109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:50.181460 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:50.181470 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.182518 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.182521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:50.182615 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:50.182777 (+ 162us) client_negotiation.cc:769] Sending connection context
0807 15:13:50.182890 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 15:13:50.183032 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":785,"mutex_wait_us":74,"thread_start_us":93,"threads_started":1}
I20260807 15:13:50.183246 9259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:50.178051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53974 (local address 127.15.226.66:44453)
0807 15:13:50.178385 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:50.178399 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:50.178573 (+ 174us) server_negotiation.cc:407] Connection header received
0807 15:13:50.178742 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:50.178746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:50.178887 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:50.179043 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:50.179238 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.179964 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.181590 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.182369 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.182487 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:50.182681 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:50.182964 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 15:13:50.183087 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":120,"threads_started":1}
I20260807 15:13:50.183892 9257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:50.177193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45326 (local address 127.15.226.67:38251)
0807 15:13:50.177609 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:50.177630 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:50.178038 (+ 408us) server_negotiation.cc:407] Connection header received
0807 15:13:50.178126 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:50.178132 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:50.178339 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:50.178495 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:50.179080 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.180305 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.182020 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.182934 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.183051 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:50.183186 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:50.183607 (+ 421us) server_negotiation.cc:299] Negotiation successful
0807 15:13:50.183719 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":148,"threads_started":1}
I20260807 15:13:50.183509 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:50.177003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:38251 (local address 127.0.0.1:45326)
0807 15:13:50.177739 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:50.177753 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:50.177888 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:50.178705 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:50.178711 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:50.178746 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:50.178942 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:50.178949 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.180490 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.180494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:50.181881 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:50.181893 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.183076 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.183079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:50.183167 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:50.183189 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 15:13:50.183281 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 15:13:50.183369 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":151,"spinlock_wait_cycles":19584,"thread_start_us":124,"threads_started":1}
I20260807 15:13:50.185068 9251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:50.176844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:38205 (local address 127.0.0.1:50170)
0807 15:13:50.177688 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:50.177740 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:50.177888 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:50.178705 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:50.178711 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:50.178744 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:50.178928 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:50.178937 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.180497 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.180500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:50.181884 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:50.181893 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.184641 (+ 2748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.184647 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:50.184755 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:50.184778 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 15:13:50.184863 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 15:13:50.184936 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":108,"spinlock_wait_cycles":2432,"thread_start_us":92,"threads_started":1}
I20260807 15:13:50.185158 9252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:50.176933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50170 (local address 127.15.226.65:38205)
0807 15:13:50.178046 (+ 1113us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:50.178055 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:50.178077 (+ 22us) server_negotiation.cc:407] Connection header received
0807 15:13:50.178144 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:50.178148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:50.178349 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:50.178495 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:50.179080 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.180378 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.183262 (+ 2884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.184489 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.184614 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:50.184761 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:50.184910 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 15:13:50.185018 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":119,"threads_started":1}
I20260807 15:13:50.187228 8911 tablet_service.cc:1467] Processing CreateTablet for tablet 2674ef566f1a41bea8a1fe5f97ddb4c2 (DEFAULT_TABLE table=test-table [id=dc39d4327c1642ea9d42f50a8cea9ef1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:13:50.187331 9131 tablet_service.cc:1467] Processing CreateTablet for tablet 2674ef566f1a41bea8a1fe5f97ddb4c2 (DEFAULT_TABLE table=test-table [id=dc39d4327c1642ea9d42f50a8cea9ef1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:13:50.188452 8911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2674ef566f1a41bea8a1fe5f97ddb4c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:50.188047 8716 tablet_service.cc:1467] Processing CreateTablet for tablet 2674ef566f1a41bea8a1fe5f97ddb4c2 (DEFAULT_TABLE table=test-table [id=dc39d4327c1642ea9d42f50a8cea9ef1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:13:50.188467 9131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2674ef566f1a41bea8a1fe5f97ddb4c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:50.189045 8716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2674ef566f1a41bea8a1fe5f97ddb4c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:50.195577 9264 tablet_bootstrap.cc:492] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1: Bootstrap starting.
I20260807 15:13:50.197242 9265 tablet_bootstrap.cc:492] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P d05c9dac1c284ac8a85aacbc92a96e06: Bootstrap starting.
I20260807 15:13:50.197806 9266 tablet_bootstrap.cc:492] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 143615220c774a93834c0844149b7a8d: Bootstrap starting.
I20260807 15:13:50.198293 9264 tablet_bootstrap.cc:654] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:50.199707 9265 tablet_bootstrap.cc:654] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P d05c9dac1c284ac8a85aacbc92a96e06: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:50.199990 9266 tablet_bootstrap.cc:654] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 143615220c774a93834c0844149b7a8d: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:50.200481 9265 log.cc:824] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P d05c9dac1c284ac8a85aacbc92a96e06: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:50.200481 9264 log.cc:824] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:50.201318 9266 log.cc:824] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 143615220c774a93834c0844149b7a8d: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:50.202860 9264 tablet_bootstrap.cc:492] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1: No bootstrap required, opened a new log
I20260807 15:13:50.202859 9265 tablet_bootstrap.cc:492] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P d05c9dac1c284ac8a85aacbc92a96e06: No bootstrap required, opened a new log
I20260807 15:13:50.203086 9264 ts_tablet_manager.cc:1397] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260807 15:13:50.203086 9265 ts_tablet_manager.cc:1397] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P d05c9dac1c284ac8a85aacbc92a96e06: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 15:13:50.203446 9266 tablet_bootstrap.cc:492] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 143615220c774a93834c0844149b7a8d: No bootstrap required, opened a new log
I20260807 15:13:50.203634 9266 ts_tablet_manager.cc:1397] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 143615220c774a93834c0844149b7a8d: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260807 15:13:50.206528 9264 raft_consensus.cc:348] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d05c9dac1c284ac8a85aacbc92a96e06" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38205 } } peers { permanent_uuid: "1d095da5fe0148aab17291957cf571c1" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44453 } } peers { permanent_uuid: "143615220c774a93834c0844149b7a8d" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 38251 } }
I20260807 15:13:50.207091 9264 raft_consensus.cc:374] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:50.207183 9264 raft_consensus.cc:729] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d095da5fe0148aab17291957cf571c1, State: Initialized, Role: FOLLOWER
I20260807 15:13:50.207682 9264 consensus_queue.cc:260] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d05c9dac1c284ac8a85aacbc92a96e06" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38205 } } peers { permanent_uuid: "1d095da5fe0148aab17291957cf571c1" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44453 } } peers { permanent_uuid: "143615220c774a93834c0844149b7a8d" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 38251 } }
I20260807 15:13:50.207777 9266 raft_consensus.cc:348] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 143615220c774a93834c0844149b7a8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d05c9dac1c284ac8a85aacbc92a96e06" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38205 } } peers { permanent_uuid: "1d095da5fe0148aab17291957cf571c1" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44453 } } peers { permanent_uuid: "143615220c774a93834c0844149b7a8d" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 38251 } }
I20260807 15:13:50.208094 9266 raft_consensus.cc:374] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 143615220c774a93834c0844149b7a8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:50.208192 9266 raft_consensus.cc:729] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 143615220c774a93834c0844149b7a8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 143615220c774a93834c0844149b7a8d, State: Initialized, Role: FOLLOWER
I20260807 15:13:50.208658 9266 consensus_queue.cc:260] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 143615220c774a93834c0844149b7a8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d05c9dac1c284ac8a85aacbc92a96e06" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38205 } } peers { permanent_uuid: "1d095da5fe0148aab17291957cf571c1" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44453 } } peers { permanent_uuid: "143615220c774a93834c0844149b7a8d" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 38251 } }
I20260807 15:13:50.208609 9265 raft_consensus.cc:348] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P d05c9dac1c284ac8a85aacbc92a96e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d05c9dac1c284ac8a85aacbc92a96e06" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38205 } } peers { permanent_uuid: "1d095da5fe0148aab17291957cf571c1" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44453 } } peers { permanent_uuid: "143615220c774a93834c0844149b7a8d" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 38251 } }
I20260807 15:13:50.208840 9264 ts_tablet_manager.cc:1428] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 15:13:50.208946 9265 raft_consensus.cc:374] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P d05c9dac1c284ac8a85aacbc92a96e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:50.209048 9265 raft_consensus.cc:729] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P d05c9dac1c284ac8a85aacbc92a96e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d05c9dac1c284ac8a85aacbc92a96e06, State: Initialized, Role: FOLLOWER
I20260807 15:13:50.209242 9015 heartbeater.cc:503] Master 127.15.226.126:34585 was elected leader, sending a full tablet report...
I20260807 15:13:50.209530 9236 heartbeater.cc:503] Master 127.15.226.126:34585 was elected leader, sending a full tablet report...
I20260807 15:13:50.209481 9265 consensus_queue.cc:260] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P d05c9dac1c284ac8a85aacbc92a96e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d05c9dac1c284ac8a85aacbc92a96e06" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38205 } } peers { permanent_uuid: "1d095da5fe0148aab17291957cf571c1" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44453 } } peers { permanent_uuid: "143615220c774a93834c0844149b7a8d" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 38251 } }
I20260807 15:13:50.209784 9266 ts_tablet_manager.cc:1428] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 143615220c774a93834c0844149b7a8d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 15:13:50.210472 9265 ts_tablet_manager.cc:1428] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P d05c9dac1c284ac8a85aacbc92a96e06: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 15:13:50.211035 8815 heartbeater.cc:503] Master 127.15.226.126:34585 was elected leader, sending a full tablet report...
I20260807 15:13:50.229835 9272 raft_consensus.cc:482] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:13:50.230099 9272 raft_consensus.cc:504] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d05c9dac1c284ac8a85aacbc92a96e06" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38205 } } peers { permanent_uuid: "1d095da5fe0148aab17291957cf571c1" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44453 } } peers { permanent_uuid: "143615220c774a93834c0844149b7a8d" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 38251 } }
I20260807 15:13:50.231451 9272 leader_election.cc:288] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d05c9dac1c284ac8a85aacbc92a96e06 (127.15.226.65:38205), 143615220c774a93834c0844149b7a8d (127.15.226.67:38251)
I20260807 15:13:50.236711 9017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:50.231663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:38205 (local address 127.15.226.66:58517)
0807 15:13:50.231826 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:50.231846 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:50.231956 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:50.232355 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:50.232361 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:50.232381 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:50.232508 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:50.232513 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.233602 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.233625 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:50.235405 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:50.235418 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.236362 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.236365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:50.236450 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:50.236473 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 15:13:50.236536 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 15:13:50.236595 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":263}
I20260807 15:13:50.236824 9252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:50.231941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:58517 (local address 127.15.226.65:38205)
0807 15:13:50.232080 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:50.232085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:50.232103 (+ 18us) server_negotiation.cc:407] Connection header received
0807 15:13:50.232169 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:50.232172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:50.232210 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:50.232323 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:50.232662 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.233479 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.235556 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.236228 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.236361 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:50.236418 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:50.236579 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 15:13:50.236669 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 15:13:50.236958 9276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:50.231761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:38251 (local address 127.15.226.66:51039)
0807 15:13:50.232173 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:50.232184 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:50.232267 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:50.232588 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:50.232593 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:50.232617 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:50.232723 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:50.232727 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.233661 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.233664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:50.235405 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:50.235418 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.236511 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.236537 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:50.236646 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:50.236669 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 15:13:50.236752 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 15:13:50.236815 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":165,"thread_start_us":168,"threads_started":1}
I20260807 15:13:50.237056 9257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:50.231941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:51039 (local address 127.15.226.67:38251)
0807 15:13:50.232080 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:50.232085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:50.232271 (+ 186us) server_negotiation.cc:407] Connection header received
0807 15:13:50.232416 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:50.232419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:50.232482 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:50.232548 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:50.232827 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.233555 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.235556 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.236190 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.236361 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:50.236418 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:50.236808 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 15:13:50.236899 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 15:13:50.237629 8748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2674ef566f1a41bea8a1fe5f97ddb4c2" candidate_uuid: "1d095da5fe0148aab17291957cf571c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d05c9dac1c284ac8a85aacbc92a96e06" is_pre_election: true
I20260807 15:13:50.237689 9162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2674ef566f1a41bea8a1fe5f97ddb4c2" candidate_uuid: "1d095da5fe0148aab17291957cf571c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "143615220c774a93834c0844149b7a8d" is_pre_election: true
I20260807 15:13:50.237965 9162 raft_consensus.cc:2454] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 143615220c774a93834c0844149b7a8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d095da5fe0148aab17291957cf571c1 in term 0.
I20260807 15:13:50.237968 8748 raft_consensus.cc:2454] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P d05c9dac1c284ac8a85aacbc92a96e06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d095da5fe0148aab17291957cf571c1 in term 0.
I20260807 15:13:50.238533 8846 leader_election.cc:302] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 143615220c774a93834c0844149b7a8d, 1d095da5fe0148aab17291957cf571c1; no voters:
I20260807 15:13:50.238900 9272 raft_consensus.cc:2781] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:13:50.239069 9272 raft_consensus.cc:482] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:13:50.239140 9272 raft_consensus.cc:3037] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:50.240334 9272 raft_consensus.cc:504] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d05c9dac1c284ac8a85aacbc92a96e06" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38205 } } peers { permanent_uuid: "1d095da5fe0148aab17291957cf571c1" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44453 } } peers { permanent_uuid: "143615220c774a93834c0844149b7a8d" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 38251 } }
I20260807 15:13:50.240749 9272 leader_election.cc:288] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1 [CANDIDATE]: Term 1 election: Requested vote from peers d05c9dac1c284ac8a85aacbc92a96e06 (127.15.226.65:38205), 143615220c774a93834c0844149b7a8d (127.15.226.67:38251)
I20260807 15:13:50.241133 8748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2674ef566f1a41bea8a1fe5f97ddb4c2" candidate_uuid: "1d095da5fe0148aab17291957cf571c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d05c9dac1c284ac8a85aacbc92a96e06"
I20260807 15:13:50.241248 9162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2674ef566f1a41bea8a1fe5f97ddb4c2" candidate_uuid: "1d095da5fe0148aab17291957cf571c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "143615220c774a93834c0844149b7a8d"
I20260807 15:13:50.241266 8748 raft_consensus.cc:3037] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P d05c9dac1c284ac8a85aacbc92a96e06 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:50.241421 9162 raft_consensus.cc:3037] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 143615220c774a93834c0844149b7a8d [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:50.242479 8748 raft_consensus.cc:2454] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P d05c9dac1c284ac8a85aacbc92a96e06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d095da5fe0148aab17291957cf571c1 in term 1.
I20260807 15:13:50.242594 9162 raft_consensus.cc:2454] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 143615220c774a93834c0844149b7a8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d095da5fe0148aab17291957cf571c1 in term 1.
I20260807 15:13:50.242866 8843 leader_election.cc:302] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d095da5fe0148aab17291957cf571c1, d05c9dac1c284ac8a85aacbc92a96e06; no voters:
I20260807 15:13:50.243075 9272 raft_consensus.cc:2781] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:13:50.243330 9272 raft_consensus.cc:686] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1 [term 1 LEADER]: Becoming Leader. State: Replica: 1d095da5fe0148aab17291957cf571c1, State: Running, Role: LEADER
I20260807 15:13:50.243695 9272 consensus_queue.cc:237] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d05c9dac1c284ac8a85aacbc92a96e06" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38205 } } peers { permanent_uuid: "1d095da5fe0148aab17291957cf571c1" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44453 } } peers { permanent_uuid: "143615220c774a93834c0844149b7a8d" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 38251 } }
I20260807 15:13:50.247051 8518 catalog_manager.cc:5495] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d095da5fe0148aab17291957cf571c1 (127.15.226.66). New cstate: current_term: 1 leader_uuid: "1d095da5fe0148aab17291957cf571c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d05c9dac1c284ac8a85aacbc92a96e06" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d095da5fe0148aab17291957cf571c1" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "143615220c774a93834c0844149b7a8d" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 38251 } health_report { overall_health: UNKNOWN } } }
I20260807 15:13:50.263612 9259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:50.260229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53990 (local address 127.15.226.66:44453)
0807 15:13:50.260373 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:50.260377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:50.260392 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:13:50.260526 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:50.260529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:50.260587 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:50.260680 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:13:50.260894 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.261399 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.262221 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.262434 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.262515 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:50.262829 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:13:50.263258 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:13:50.263353 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:50.263425 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 15:13:50.263476 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 15:13:50.269874 8748 raft_consensus.cc:1261] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P d05c9dac1c284ac8a85aacbc92a96e06 [term 1 FOLLOWER]: Refusing update from remote peer 1d095da5fe0148aab17291957cf571c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:13:50.269874 9162 raft_consensus.cc:1261] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 143615220c774a93834c0844149b7a8d [term 1 FOLLOWER]: Refusing update from remote peer 1d095da5fe0148aab17291957cf571c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:13:50.270738 9277 consensus_queue.cc:1035] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d05c9dac1c284ac8a85aacbc92a96e06" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 38205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:13:50.270978 9272 consensus_queue.cc:1035] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "143615220c774a93834c0844149b7a8d" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 38251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:13:50.279597 9285 mvcc.cc:205] Tried to move back new op lower bound from 7315929621577474048 to 7315929621481021440. Current Snapshot: MvccSnapshot[applied={T|T < 7315929621577474048}]
I20260807 15:13:50.280121 9284 mvcc.cc:205] Tried to move back new op lower bound from 7315929621577474048 to 7315929621481021440. Current Snapshot: MvccSnapshot[applied={T|T < 7315929621577474048}]
I20260807 15:13:50.291882 8518 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:13:50.292102 8518 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:13:50.294894 8518 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e8040d87defd4f6696094d9f5ffcdea4: Sending DeleteTablet for 3 replicas of tablet 2674ef566f1a41bea8a1fe5f97ddb4c2
I20260807 15:13:50.295892 9131 tablet_service.cc:1514] Processing DeleteTablet for tablet 2674ef566f1a41bea8a1fe5f97ddb4c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:13:50 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:45326
I20260807 15:13:50.296146 8911 tablet_service.cc:1514] Processing DeleteTablet for tablet 2674ef566f1a41bea8a1fe5f97ddb4c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:13:50 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:53974
I20260807 15:13:50.296165 8716 tablet_service.cc:1514] Processing DeleteTablet for tablet 2674ef566f1a41bea8a1fe5f97ddb4c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:13:50 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:50170
I20260807 15:13:50.296770 9301 tablet_replica.cc:330] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P d05c9dac1c284ac8a85aacbc92a96e06: stopping tablet replica
I20260807 15:13:50.296784 9298 tablet_replica.cc:330] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 143615220c774a93834c0844149b7a8d: stopping tablet replica
I20260807 15:13:50.297154 9298 raft_consensus.cc:2229] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 143615220c774a93834c0844149b7a8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:13:50.297163 9301 raft_consensus.cc:2229] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P d05c9dac1c284ac8a85aacbc92a96e06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:13:50.297303 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 8620
I20260807 15:13:50.297575 9301 raft_consensus.cc:2258] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P d05c9dac1c284ac8a85aacbc92a96e06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:13:50.300146 9298 raft_consensus.cc:2258] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 143615220c774a93834c0844149b7a8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:13:50.303968 9298 ts_tablet_manager.cc:1905] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 143615220c774a93834c0844149b7a8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:13:50.307345 9298 ts_tablet_manager.cc:1918] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 143615220c774a93834c0844149b7a8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:13:50.307567 9298 log.cc:1196] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 143615220c774a93834c0844149b7a8d: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/2674ef566f1a41bea8a1fe5f97ddb4c2
I20260807 15:13:50.308061 9298 ts_tablet_manager.cc:1939] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 143615220c774a93834c0844149b7a8d: Deleting consensus metadata
I20260807 15:13:50.318990 9299 tablet_replica.cc:330] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1: stopping tablet replica
I20260807 15:13:50.319594 9299 raft_consensus.cc:2229] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:13:50.320294 9299 raft_consensus.cc:2258] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:13:50.320595 8500 catalog_manager.cc:4841] TS 143615220c774a93834c0844149b7a8d (127.15.226.67:38251): tablet 2674ef566f1a41bea8a1fe5f97ddb4c2 (table test-table [id=dc39d4327c1642ea9d42f50a8cea9ef1]) successfully deleted
I20260807 15:13:50.322168 9299 ts_tablet_manager.cc:1905] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:13:50.325399 9299 ts_tablet_manager.cc:1918] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:13:50.325525 9299 log.cc:1196] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/2674ef566f1a41bea8a1fe5f97ddb4c2
I20260807 15:13:50.325945 9299 ts_tablet_manager.cc:1939] T 2674ef566f1a41bea8a1fe5f97ddb4c2 P 1d095da5fe0148aab17291957cf571c1: Deleting consensus metadata
I20260807 15:13:50.327412 8498 catalog_manager.cc:4841] TS 1d095da5fe0148aab17291957cf571c1 (127.15.226.66:44453): tablet 2674ef566f1a41bea8a1fe5f97ddb4c2 (table test-table [id=dc39d4327c1642ea9d42f50a8cea9ef1]) successfully deleted
W20260807 15:13:50.327875 8498 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.226.65:38205 (error 108)
I20260807 15:13:50.328193 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 8823
I20260807 15:13:50.330907 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:50.328883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:38205 (local address 127.0.0.1:50172)
0807 15:13:50.330627 (+ 1744us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:50.330778 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.65:38205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1617}
W20260807 15:13:50.331229 8498 catalog_manager.cc:4584] TS d05c9dac1c284ac8a85aacbc92a96e06 (127.15.226.65:38205): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2674ef566f1a41bea8a1fe5f97ddb4c2: Network error: Client connection negotiation failed: client connection to 127.15.226.65:38205: connect: Connection refused (error 111)
I20260807 15:13:50.346468 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 9022
I20260807 15:13:50.356170 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 8473
2026-08-07T15:13:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2649 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T15:13:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:13:50Z Disabled control of system clock
I20260807 15:13:50.395871 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:32779
--webserver_interface=127.15.226.126
--webserver_port=0
--builtin_ntp_servers=127.15.226.84:39211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:32779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:13:50.513502 9308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:50.513808 9308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:50.513846 9308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:50.517866 9308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:13:50.517958 9308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:50.517973 9308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:50.517992 9308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:13:50.518006 9308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:13:50.523123 9308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:39211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:32779
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:32779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:50.524291 9308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:50.525785 9308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:50.534991 9314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:50.535049 9315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:50.535049 9317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:50.535029 9308 server_base.cc:1034] running on GCE node
I20260807 15:13:50.536005 9308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:50.537161 9308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:50.538364 9308 hybrid_clock.cc:648] HybridClock initialized: now 1786115630538338 us; error 43 us; skew 500 ppm
I20260807 15:13:50.540838 9308 webserver.cc:460] Webserver started at http://127.15.226.126:39005/ using document root <none> and password file <none>
I20260807 15:13:50.541501 9308 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:50.541576 9308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:50.541843 9308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:13:50.544178 9308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "2cea7f749c26495fb51984a246c156b1"
format_stamp: "Formatted at 2026-08-07 15:13:50 on dist-test-slave-3kk6"
server_key: "2fdd4ab3b1e6ea37179e20ca36ebf5ee"
server_key_iv: "a0bc71dea8d798612c41aad28da45861"
server_key_version: "encryptionkey@0"
I20260807 15:13:50.544813 9308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "2cea7f749c26495fb51984a246c156b1"
format_stamp: "Formatted at 2026-08-07 15:13:50 on dist-test-slave-3kk6"
server_key: "2fdd4ab3b1e6ea37179e20ca36ebf5ee"
server_key_iv: "a0bc71dea8d798612c41aad28da45861"
server_key_version: "encryptionkey@0"
I20260807 15:13:50.548862 9308 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:13:50.551623 9324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:50.553151 9308 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:13:50.553303 9308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "2cea7f749c26495fb51984a246c156b1"
format_stamp: "Formatted at 2026-08-07 15:13:50 on dist-test-slave-3kk6"
server_key: "2fdd4ab3b1e6ea37179e20ca36ebf5ee"
server_key_iv: "a0bc71dea8d798612c41aad28da45861"
server_key_version: "encryptionkey@0"
I20260807 15:13:50.553439 9308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:50.576103 9308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:50.576939 9308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:50.577172 9308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:50.585084 9308 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:32779
I20260807 15:13:50.585106 9392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:32779 every 8 connection(s)
I20260807 15:13:50.586098 9308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:13:50.589447 9396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:50.593344 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 9308
I20260807 15:13:50.593492 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:13:50.593829 16265 external_mini_cluster.cc:1428] Setting key 05f760999bccc01d3db40ae01cc1dfc4
I20260807 15:13:50.596122 9396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cea7f749c26495fb51984a246c156b1: Bootstrap starting.
I20260807 15:13:50.598433 9396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cea7f749c26495fb51984a246c156b1: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:50.599258 9396 log.cc:824] T 00000000000000000000000000000000 P 2cea7f749c26495fb51984a246c156b1: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:50.601562 9396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cea7f749c26495fb51984a246c156b1: No bootstrap required, opened a new log
I20260807 15:13:50.604295 9401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:50.595516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43150 (local address 127.15.226.126:32779)
0807 15:13:50.596282 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:50.596301 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:50.596343 (+ 42us) server_negotiation.cc:407] Connection header received
0807 15:13:50.597040 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:50.597058 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:50.597466 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:50.597816 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:50.598300 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.599450 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.600638 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.601154 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.601355 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:50.602352 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:50.602357 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:50.602371 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:13:50.602874 (+ 503us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:50.602940 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:13:50.602963 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:50.603079 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:50.603209 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 15:13:50.603521 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":169,"threads_started":1}
I20260807 15:13:50.605759 9396 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cea7f749c26495fb51984a246c156b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cea7f749c26495fb51984a246c156b1" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 32779 } }
I20260807 15:13:50.606036 9396 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2cea7f749c26495fb51984a246c156b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:50.606122 9396 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cea7f749c26495fb51984a246c156b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cea7f749c26495fb51984a246c156b1, State: Initialized, Role: FOLLOWER
I20260807 15:13:50.606637 9396 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cea7f749c26495fb51984a246c156b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cea7f749c26495fb51984a246c156b1" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 32779 } }
I20260807 15:13:50.606865 9396 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2cea7f749c26495fb51984a246c156b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:13:50.606992 9396 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2cea7f749c26495fb51984a246c156b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:13:50.607125 9396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cea7f749c26495fb51984a246c156b1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:50.608166 9396 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2cea7f749c26495fb51984a246c156b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cea7f749c26495fb51984a246c156b1" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 32779 } }
I20260807 15:13:50.608536 9396 leader_election.cc:302] T 00000000000000000000000000000000 P 2cea7f749c26495fb51984a246c156b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2cea7f749c26495fb51984a246c156b1; no voters:
I20260807 15:13:50.608927 9396 leader_election.cc:288] T 00000000000000000000000000000000 P 2cea7f749c26495fb51984a246c156b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:13:50.609061 9404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2cea7f749c26495fb51984a246c156b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:13:50.609642 9404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2cea7f749c26495fb51984a246c156b1 [term 1 LEADER]: Becoming Leader. State: Replica: 2cea7f749c26495fb51984a246c156b1, State: Running, Role: LEADER
I20260807 15:13:50.610028 9404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cea7f749c26495fb51984a246c156b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cea7f749c26495fb51984a246c156b1" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 32779 } }
I20260807 15:13:50.610363 9396 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cea7f749c26495fb51984a246c156b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:13:50.611243 9404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cea7f749c26495fb51984a246c156b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cea7f749c26495fb51984a246c156b1. Latest consensus state: current_term: 1 leader_uuid: "2cea7f749c26495fb51984a246c156b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cea7f749c26495fb51984a246c156b1" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 32779 } } }
I20260807 15:13:50.611495 9404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cea7f749c26495fb51984a246c156b1 [sys.catalog]: This master's current role is: LEADER
I20260807 15:13:50.611672 9405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cea7f749c26495fb51984a246c156b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2cea7f749c26495fb51984a246c156b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cea7f749c26495fb51984a246c156b1" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 32779 } } }
I20260807 15:13:50.611792 9405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cea7f749c26495fb51984a246c156b1 [sys.catalog]: This master's current role is: LEADER
I20260807 15:13:50.612099 9419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:13:50.615443 9419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:13:50.621551 9419 catalog_manager.cc:1347] Generated new cluster ID: a832525c456b4fadbf49cd7f140d1b57
I20260807 15:13:50.621641 9419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:13:50.633286 9419 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:13:50.634261 9419 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:13:50.642740 9419 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2cea7f749c26495fb51984a246c156b1: Generated new TSK 0
I20260807 15:13:50.643510 9419 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:13:50.648989 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:32779
--builtin_ntp_servers=127.15.226.84:39211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:13:50.773169 9432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:50.773492 9432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:50.773535 9432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:50.777707 9432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:50.777787 9432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:50.777846 9432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:13:50.782862 9432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:39211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:32779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:50.784041 9432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:50.785549 9432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:50.796205 9442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:50.796205 9440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:50.796304 9439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:50.796491 9432 server_base.cc:1034] running on GCE node
I20260807 15:13:50.797093 9432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:50.797772 9432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:50.798949 9432 hybrid_clock.cc:648] HybridClock initialized: now 1786115630798914 us; error 57 us; skew 500 ppm
I20260807 15:13:50.801268 9432 webserver.cc:460] Webserver started at http://127.15.226.65:37317/ using document root <none> and password file <none>
I20260807 15:13:50.801867 9432 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:50.801935 9432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:50.802124 9432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:13:50.804420 9432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "0b9da635d5b645f384c8068d821b0881"
format_stamp: "Formatted at 2026-08-07 15:13:50 on dist-test-slave-3kk6"
server_key: "fd5941c34f6f98451663f16f45b90112"
server_key_iv: "184de78a59745431080042308f81c81e"
server_key_version: "encryptionkey@0"
I20260807 15:13:50.805038 9432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "0b9da635d5b645f384c8068d821b0881"
format_stamp: "Formatted at 2026-08-07 15:13:50 on dist-test-slave-3kk6"
server_key: "fd5941c34f6f98451663f16f45b90112"
server_key_iv: "184de78a59745431080042308f81c81e"
server_key_version: "encryptionkey@0"
I20260807 15:13:50.809139 9432 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260807 15:13:50.812145 9452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:50.813535 9432 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:13:50.813687 9432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "0b9da635d5b645f384c8068d821b0881"
format_stamp: "Formatted at 2026-08-07 15:13:50 on dist-test-slave-3kk6"
server_key: "fd5941c34f6f98451663f16f45b90112"
server_key_iv: "184de78a59745431080042308f81c81e"
server_key_version: "encryptionkey@0"
I20260807 15:13:50.813832 9432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:50.823776 9432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:50.824678 9432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:50.824929 9432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:50.825620 9432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:50.826821 9432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:50.827005 9432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:50.827129 9432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:50.827188 9432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:50.837563 9432 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:33265
I20260807 15:13:50.837617 9632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:33265 every 8 connection(s)
I20260807 15:13:50.838559 9432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:13:50.847712 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 9432
I20260807 15:13:50.847878 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:13:50.848284 16265 external_mini_cluster.cc:1428] Setting key d7736be96545b26f3c49db456f932b38
I20260807 15:13:50.849028 9401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:50.840676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:41333 (local address 127.15.226.126:32779)
0807 15:13:50.840866 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:50.840870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:50.842015 (+ 1145us) server_negotiation.cc:407] Connection header received
0807 15:13:50.843575 (+ 1560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:50.843578 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:50.843642 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:50.843732 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:50.844747 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.845328 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.846534 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.846855 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.846974 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:50.848390 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:50.848395 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:50.848397 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:13:50.848414 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:50.848444 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:13:50.848454 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:50.848517 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:50.848720 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 15:13:50.848810 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 15:13:50.850124 9635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:50.841074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:32779 (local address 127.15.226.65:41333)
0807 15:13:50.841748 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:50.841792 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:50.843337 (+ 1545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:50.843932 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:50.843939 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:50.844026 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:50.844579 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:50.844593 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.845473 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.845483 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:50.846394 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:50.846404 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:50.847046 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:50.847053 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:50.847218 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:50.848011 (+ 793us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:13:50.848014 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:50.848077 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:13:50.848186 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:50.848548 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:50.848559 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:50.848566 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 15:13:50.848725 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 15:13:50.849038 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":127,"threads_started":1}
I20260807 15:13:50.850937 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:32779
--builtin_ntp_servers=127.15.226.84:39211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 15:13:50.851442 9633 heartbeater.cc:347] Connected to a master server at 127.15.226.126:32779
I20260807 15:13:50.851744 9633 heartbeater.cc:464] Registering TS with master...
I20260807 15:13:50.852430 9633 heartbeater.cc:511] Master 127.15.226.126:32779 requested a full tablet report, sending...
I20260807 15:13:50.854372 9350 ts_manager.cc:194] Registered new tserver with Master: 0b9da635d5b645f384c8068d821b0881 (127.15.226.65:33265)
I20260807 15:13:50.855681 9350 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.226.65:41333
W20260807 15:13:50.971917 9636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:50.972206 9636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:50.972249 9636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:50.976423 9636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:50.976485 9636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:50.976565 9636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:13:50.981604 9636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:39211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:32779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:50.982859 9636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:50.984303 9636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:50.994558 9646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:50.994602 9636 server_base.cc:1034] running on GCE node
W20260807 15:13:50.994544 9648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:50.994544 9645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:50.995355 9636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:50.996060 9636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:50.997231 9636 hybrid_clock.cc:648] HybridClock initialized: now 1786115630997204 us; error 53 us; skew 500 ppm
I20260807 15:13:51.004907 9636 webserver.cc:460] Webserver started at http://127.15.226.66:42805/ using document root <none> and password file <none>
I20260807 15:13:51.005600 9636 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:51.005673 9636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:51.005914 9636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:13:51.008306 9636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "c59cb48711aa45fc8e1b4bafb3ec8d36"
format_stamp: "Formatted at 2026-08-07 15:13:51 on dist-test-slave-3kk6"
server_key: "0012fcefb4f31871f2a67744dbfec384"
server_key_iv: "5444c6229e9ab35d9a86421298e98bd8"
server_key_version: "encryptionkey@0"
I20260807 15:13:51.008947 9636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "c59cb48711aa45fc8e1b4bafb3ec8d36"
format_stamp: "Formatted at 2026-08-07 15:13:51 on dist-test-slave-3kk6"
server_key: "0012fcefb4f31871f2a67744dbfec384"
server_key_iv: "5444c6229e9ab35d9a86421298e98bd8"
server_key_version: "encryptionkey@0"
I20260807 15:13:51.013227 9636 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:13:51.015939 9658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:51.017328 9636 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:13:51.017454 9636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "c59cb48711aa45fc8e1b4bafb3ec8d36"
format_stamp: "Formatted at 2026-08-07 15:13:51 on dist-test-slave-3kk6"
server_key: "0012fcefb4f31871f2a67744dbfec384"
server_key_iv: "5444c6229e9ab35d9a86421298e98bd8"
server_key_version: "encryptionkey@0"
I20260807 15:13:51.017566 9636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:51.041136 9636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:51.041951 9636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:51.042181 9636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:51.042872 9636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:51.043915 9636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:51.043990 9636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:51.044065 9636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:51.044109 9636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:51.053956 9636 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:38551
I20260807 15:13:51.053984 9836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:38551 every 8 connection(s)
I20260807 15:13:51.055052 9636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:13:51.060333 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 9636
I20260807 15:13:51.060496 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:13:51.060817 16265 external_mini_cluster.cc:1428] Setting key 2a38d6c59ed9325bd88c5d6ef1d4e9ae
I20260807 15:13:51.062989 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:32779
--builtin_ntp_servers=127.15.226.84:39211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 15:13:51.065536 9401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:51.057084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:36277 (local address 127.15.226.126:32779)
0807 15:13:51.057230 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:51.057233 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:51.058365 (+ 1132us) server_negotiation.cc:407] Connection header received
0807 15:13:51.059925 (+ 1560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:51.059928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:51.060014 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:51.060130 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:51.061214 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.061774 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.063296 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.063675 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.063814 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:51.065015 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:51.065019 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:51.065022 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:13:51.065041 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:51.065062 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:13:51.065070 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:51.065151 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:51.065315 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 15:13:51.065389 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:13:51.066586 9842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:51.057529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:32779 (local address 127.15.226.66:36277)
0807 15:13:51.058205 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:51.058250 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:51.059675 (+ 1425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:51.060298 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:51.060305 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:51.060407 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:51.061050 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:51.061067 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.061912 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.061916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:51.063090 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:51.063121 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.063864 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.063867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:51.064037 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:51.064722 (+ 685us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:13:51.064724 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:51.064768 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:13:51.064847 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:51.065154 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:51.065167 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:51.065173 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:13:51.065315 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 15:13:51.065627 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":138,"threads_started":1}
I20260807 15:13:51.067904 9837 heartbeater.cc:347] Connected to a master server at 127.15.226.126:32779
I20260807 15:13:51.068174 9837 heartbeater.cc:464] Registering TS with master...
I20260807 15:13:51.068811 9837 heartbeater.cc:511] Master 127.15.226.126:32779 requested a full tablet report, sending...
I20260807 15:13:51.070065 9350 ts_manager.cc:194] Registered new tserver with Master: c59cb48711aa45fc8e1b4bafb3ec8d36 (127.15.226.66:38551)
I20260807 15:13:51.070775 9350 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.226.66:36277
W20260807 15:13:51.188122 9844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:51.188433 9844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:51.188475 9844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:51.192854 9844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:51.192921 9844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:51.193002 9844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:13:51.197921 9844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:39211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:32779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:51.199186 9844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:51.200651 9844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:51.211910 9851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:51.211917 9850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:51.212020 9844 server_base.cc:1034] running on GCE node
W20260807 15:13:51.211910 9856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:51.212708 9844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:51.213459 9844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:51.214701 9844 hybrid_clock.cc:648] HybridClock initialized: now 1786115631214626 us; error 33 us; skew 500 ppm
I20260807 15:13:51.216905 9844 webserver.cc:460] Webserver started at http://127.15.226.67:37759/ using document root <none> and password file <none>
I20260807 15:13:51.217609 9844 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:51.217739 9844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:51.217990 9844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:13:51.220980 9844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "544dd2b84c8e4efbb48d05b4be836234"
format_stamp: "Formatted at 2026-08-07 15:13:51 on dist-test-slave-3kk6"
server_key: "02f94f47eaf4b9cc059031735705fea8"
server_key_iv: "21a00f9d2fde09796db45134d0f9d50d"
server_key_version: "encryptionkey@0"
I20260807 15:13:51.221691 9844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "544dd2b84c8e4efbb48d05b4be836234"
format_stamp: "Formatted at 2026-08-07 15:13:51 on dist-test-slave-3kk6"
server_key: "02f94f47eaf4b9cc059031735705fea8"
server_key_iv: "21a00f9d2fde09796db45134d0f9d50d"
server_key_version: "encryptionkey@0"
I20260807 15:13:51.226207 9844 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260807 15:13:51.229504 9864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:51.231024 9844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:13:51.231199 9844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "544dd2b84c8e4efbb48d05b4be836234"
format_stamp: "Formatted at 2026-08-07 15:13:51 on dist-test-slave-3kk6"
server_key: "02f94f47eaf4b9cc059031735705fea8"
server_key_iv: "21a00f9d2fde09796db45134d0f9d50d"
server_key_version: "encryptionkey@0"
I20260807 15:13:51.231345 9844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:51.247792 9844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:51.248490 9844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:51.248657 9844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:51.249220 9844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:51.250142 9844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:51.250191 9844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:51.250233 9844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:51.250257 9844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:51.260512 9844 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:46401
I20260807 15:13:51.260560 10022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:46401 every 8 connection(s)
I20260807 15:13:51.261514 9844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:13:51.262264 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 9844
I20260807 15:13:51.262359 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:13:51.262799 16265 external_mini_cluster.cc:1428] Setting key 28d3656dc0de93e62fba1b597d2fd482
I20260807 15:13:51.272399 9401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:51.263684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:48549 (local address 127.15.226.126:32779)
0807 15:13:51.263861 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:51.263866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:51.265609 (+ 1743us) server_negotiation.cc:407] Connection header received
0807 15:13:51.267237 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:51.267240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:51.267304 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:51.267386 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:51.268351 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.268921 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.270166 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.270412 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.270533 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:51.271845 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:51.271849 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:51.271852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:13:51.271870 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:51.271897 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:13:51.271905 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:51.271995 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:51.272167 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 15:13:51.272239 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:13:51.273402 10026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:51.264191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:32779 (local address 127.15.226.67:48549)
0807 15:13:51.265458 (+ 1267us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:51.265502 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:51.266989 (+ 1487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:51.267576 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:51.267584 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:51.267675 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:51.268200 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:51.268215 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.269068 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.269072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:51.270022 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:51.270033 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.270600 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.270605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:51.270840 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:51.271549 (+ 709us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:13:51.271550 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:51.271595 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:13:51.271664 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:51.272010 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:51.272027 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:51.272033 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:13:51.272168 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 15:13:51.272504 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"thread_start_us":150,"threads_started":1}
I20260807 15:13:51.274762 10023 heartbeater.cc:347] Connected to a master server at 127.15.226.126:32779
I20260807 15:13:51.275098 10023 heartbeater.cc:464] Registering TS with master...
I20260807 15:13:51.275702 10023 heartbeater.cc:511] Master 127.15.226.126:32779 requested a full tablet report, sending...
I20260807 15:13:51.276927 9349 ts_manager.cc:194] Registered new tserver with Master: 544dd2b84c8e4efbb48d05b4be836234 (127.15.226.67:46401)
I20260807 15:13:51.277555 9349 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.226.67:48549
I20260807 15:13:51.289495 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:13:51.296068 9401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:51.291979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43160 (local address 127.15.226.126:32779)
0807 15:13:51.292140 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:51.292146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:51.292381 (+ 235us) server_negotiation.cc:407] Connection header received
0807 15:13:51.292487 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:51.292489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:51.292537 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:51.292612 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:51.292828 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.293358 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.294204 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.294415 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.294510 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:51.295504 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:51.295507 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:51.295509 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:13:51.295522 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:51.295542 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:13:51.295551 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:51.295614 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:51.295798 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 15:13:51.295854 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 15:13:51.299472 9349 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43160:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:13:51.303341 9349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:13:51.319924 10043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:51.313446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:38551 (local address 127.0.0.1:51700)
0807 15:13:51.314892 (+ 1446us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:51.314907 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:51.315063 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:51.315683 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:51.315685 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:51.315702 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:51.315795 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:51.315799 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.316943 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.316946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:51.318228 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:51.318239 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.319298 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.319305 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:51.319420 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:51.319473 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 15:13:51.319597 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 15:13:51.319736 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1380,"mutex_wait_us":186,"thread_start_us":93,"threads_started":1}
I20260807 15:13:51.320261 10048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:51.314034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57812 (local address 127.15.226.67:46401)
0807 15:13:51.314503 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:51.314513 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:51.314960 (+ 447us) server_negotiation.cc:407] Connection header received
0807 15:13:51.315075 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:51.315081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:51.315321 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:51.315493 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:51.315917 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.317030 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.318611 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.319265 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.319373 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:51.319574 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:51.319971 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 15:13:51.320107 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":203,"threads_started":1}
I20260807 15:13:51.320640 10047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:51.313875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51700 (local address 127.15.226.66:38551)
0807 15:13:51.314502 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:51.314515 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:51.314992 (+ 477us) server_negotiation.cc:407] Connection header received
0807 15:13:51.315210 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:51.315216 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:51.315427 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:51.315538 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:51.315915 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.316811 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.318362 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.319155 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.319283 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:51.319500 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:51.320349 (+ 849us) server_negotiation.cc:299] Negotiation successful
0807 15:13:51.320480 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":120,"threads_started":1}
I20260807 15:13:51.320129 10044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:51.313043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:33265 (local address 127.0.0.1:51186)
0807 15:13:51.314245 (+ 1202us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:51.314306 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:51.314529 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:51.315131 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:51.315138 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:51.315163 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:51.315317 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:51.315324 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.317032 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.317036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:51.318542 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:51.318552 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.319603 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.319607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:51.319693 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:51.319725 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 15:13:51.319793 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 15:13:51.319867 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1008,"mutex_wait_us":44,"thread_start_us":107,"threads_started":1}
I20260807 15:13:51.321045 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:51.313224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51186 (local address 127.15.226.65:33265)
0807 15:13:51.313947 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:51.313955 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:51.314406 (+ 451us) server_negotiation.cc:407] Connection header received
0807 15:13:51.314751 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:51.314756 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:51.314962 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:51.315142 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:51.315456 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.316850 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.318701 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.319485 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.319594 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:51.319771 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:51.320681 (+ 910us) server_negotiation.cc:299] Negotiation successful
0807 15:13:51.320823 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":95,"threads_started":1}
I20260807 15:13:51.320536 10046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:51.313222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:46401 (local address 127.0.0.1:57812)
0807 15:13:51.314836 (+ 1614us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:51.314851 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:51.314958 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:51.315654 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:51.315658 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:51.315676 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:51.315779 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:51.315785 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.317143 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.317146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:51.318459 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:51.318471 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.319638 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.319641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:51.319726 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:51.319747 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 15:13:51.319817 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 15:13:51.319877 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1520,"mutex_wait_us":66,"thread_start_us":102,"threads_started":1}
I20260807 15:13:51.324306 9532 tablet_service.cc:1467] Processing CreateTablet for tablet 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba (DEFAULT_TABLE table=test-table [id=74b5226284224ce69d33636680cc05a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:13:51.324312 9727 tablet_service.cc:1467] Processing CreateTablet for tablet 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba (DEFAULT_TABLE table=test-table [id=74b5226284224ce69d33636680cc05a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:13:51.324805 9930 tablet_service.cc:1467] Processing CreateTablet for tablet 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba (DEFAULT_TABLE table=test-table [id=74b5226284224ce69d33636680cc05a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:13:51.325544 9532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:51.325544 9727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:51.325745 9930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:51.332404 10051 tablet_bootstrap.cc:492] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234: Bootstrap starting.
I20260807 15:13:51.333885 10052 tablet_bootstrap.cc:492] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P c59cb48711aa45fc8e1b4bafb3ec8d36: Bootstrap starting.
I20260807 15:13:51.335042 10053 tablet_bootstrap.cc:492] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 0b9da635d5b645f384c8068d821b0881: Bootstrap starting.
I20260807 15:13:51.335099 10051 tablet_bootstrap.cc:654] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:51.336011 10051 log.cc:824] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:51.336469 10052 tablet_bootstrap.cc:654] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P c59cb48711aa45fc8e1b4bafb3ec8d36: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:51.337318 10053 tablet_bootstrap.cc:654] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 0b9da635d5b645f384c8068d821b0881: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:51.337431 10052 log.cc:824] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P c59cb48711aa45fc8e1b4bafb3ec8d36: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:51.338152 10053 log.cc:824] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 0b9da635d5b645f384c8068d821b0881: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:51.338342 10051 tablet_bootstrap.cc:492] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234: No bootstrap required, opened a new log
I20260807 15:13:51.338613 10051 ts_tablet_manager.cc:1397] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 15:13:51.339880 10052 tablet_bootstrap.cc:492] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P c59cb48711aa45fc8e1b4bafb3ec8d36: No bootstrap required, opened a new log
I20260807 15:13:51.340088 10052 ts_tablet_manager.cc:1397] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P c59cb48711aa45fc8e1b4bafb3ec8d36: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 15:13:51.340278 10053 tablet_bootstrap.cc:492] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 0b9da635d5b645f384c8068d821b0881: No bootstrap required, opened a new log
I20260807 15:13:51.340481 10053 ts_tablet_manager.cc:1397] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 0b9da635d5b645f384c8068d821b0881: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 15:13:51.342908 10051 raft_consensus.cc:348] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b9da635d5b645f384c8068d821b0881" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 33265 } } peers { permanent_uuid: "544dd2b84c8e4efbb48d05b4be836234" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46401 } } peers { permanent_uuid: "c59cb48711aa45fc8e1b4bafb3ec8d36" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 38551 } }
I20260807 15:13:51.343186 10051 raft_consensus.cc:374] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:51.343255 10051 raft_consensus.cc:729] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 544dd2b84c8e4efbb48d05b4be836234, State: Initialized, Role: FOLLOWER
I20260807 15:13:51.343747 10051 consensus_queue.cc:260] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b9da635d5b645f384c8068d821b0881" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 33265 } } peers { permanent_uuid: "544dd2b84c8e4efbb48d05b4be836234" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46401 } } peers { permanent_uuid: "c59cb48711aa45fc8e1b4bafb3ec8d36" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 38551 } }
I20260807 15:13:51.344503 10023 heartbeater.cc:503] Master 127.15.226.126:32779 was elected leader, sending a full tablet report...
I20260807 15:13:51.344635 10052 raft_consensus.cc:348] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P c59cb48711aa45fc8e1b4bafb3ec8d36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b9da635d5b645f384c8068d821b0881" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 33265 } } peers { permanent_uuid: "544dd2b84c8e4efbb48d05b4be836234" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46401 } } peers { permanent_uuid: "c59cb48711aa45fc8e1b4bafb3ec8d36" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 38551 } }
I20260807 15:13:51.344843 10051 ts_tablet_manager.cc:1428] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260807 15:13:51.344906 10052 raft_consensus.cc:374] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P c59cb48711aa45fc8e1b4bafb3ec8d36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:51.344977 10052 raft_consensus.cc:729] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P c59cb48711aa45fc8e1b4bafb3ec8d36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c59cb48711aa45fc8e1b4bafb3ec8d36, State: Initialized, Role: FOLLOWER
I20260807 15:13:51.345422 10052 consensus_queue.cc:260] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P c59cb48711aa45fc8e1b4bafb3ec8d36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b9da635d5b645f384c8068d821b0881" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 33265 } } peers { permanent_uuid: "544dd2b84c8e4efbb48d05b4be836234" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46401 } } peers { permanent_uuid: "c59cb48711aa45fc8e1b4bafb3ec8d36" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 38551 } }
I20260807 15:13:51.345799 10053 raft_consensus.cc:348] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 0b9da635d5b645f384c8068d821b0881 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b9da635d5b645f384c8068d821b0881" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 33265 } } peers { permanent_uuid: "544dd2b84c8e4efbb48d05b4be836234" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46401 } } peers { permanent_uuid: "c59cb48711aa45fc8e1b4bafb3ec8d36" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 38551 } }
I20260807 15:13:51.346071 10053 raft_consensus.cc:374] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 0b9da635d5b645f384c8068d821b0881 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:51.346107 9837 heartbeater.cc:503] Master 127.15.226.126:32779 was elected leader, sending a full tablet report...
I20260807 15:13:51.346125 10053 raft_consensus.cc:729] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 0b9da635d5b645f384c8068d821b0881 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b9da635d5b645f384c8068d821b0881, State: Initialized, Role: FOLLOWER
I20260807 15:13:51.346539 10053 consensus_queue.cc:260] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 0b9da635d5b645f384c8068d821b0881 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b9da635d5b645f384c8068d821b0881" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 33265 } } peers { permanent_uuid: "544dd2b84c8e4efbb48d05b4be836234" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46401 } } peers { permanent_uuid: "c59cb48711aa45fc8e1b4bafb3ec8d36" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 38551 } }
I20260807 15:13:51.347432 10052 ts_tablet_manager.cc:1428] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P c59cb48711aa45fc8e1b4bafb3ec8d36: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 15:13:51.347640 10053 ts_tablet_manager.cc:1428] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 0b9da635d5b645f384c8068d821b0881: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 15:13:51.347899 9633 heartbeater.cc:503] Master 127.15.226.126:32779 was elected leader, sending a full tablet report...
W20260807 15:13:51.512598 10024 tablet.cc:2369] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:13:51.556375 9839 tablet.cc:2369] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P c59cb48711aa45fc8e1b4bafb3ec8d36: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:13:51.590134 9634 tablet.cc:2369] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 0b9da635d5b645f384c8068d821b0881: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:13:51.798070 10066 raft_consensus.cc:482] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:13:51.798375 10066 raft_consensus.cc:504] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b9da635d5b645f384c8068d821b0881" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 33265 } } peers { permanent_uuid: "544dd2b84c8e4efbb48d05b4be836234" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46401 } } peers { permanent_uuid: "c59cb48711aa45fc8e1b4bafb3ec8d36" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 38551 } }
I20260807 15:13:51.799710 10066 leader_election.cc:288] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b9da635d5b645f384c8068d821b0881 (127.15.226.65:33265), c59cb48711aa45fc8e1b4bafb3ec8d36 (127.15.226.66:38551)
I20260807 15:13:51.805485 10047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:51.800169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:47727 (local address 127.15.226.66:38551)
0807 15:13:51.800316 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:51.800321 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:51.800508 (+ 187us) server_negotiation.cc:407] Connection header received
0807 15:13:51.800679 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:51.800682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:51.800742 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:51.800848 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:51.801344 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.802147 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.804156 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.804773 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.804916 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:51.804970 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:51.805260 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 15:13:51.805353 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 15:13:51.805485 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:51.800185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:41063 (local address 127.15.226.65:33265)
0807 15:13:51.800335 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:51.800340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:51.800618 (+ 278us) server_negotiation.cc:407] Connection header received
0807 15:13:51.800687 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:51.800690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:51.800742 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:51.800885 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:51.801335 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.802181 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.804156 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.804783 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.804916 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:51.804970 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:51.805260 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 15:13:51.805342 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 15:13:51.805539 10069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:51.800042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:38551 (local address 127.15.226.67:47727)
0807 15:13:51.800392 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:51.800409 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:51.800550 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:51.800914 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:51.800917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:51.800957 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:51.801189 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:51.801199 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.802301 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.802304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:51.803994 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:51.804007 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.804931 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.804934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:51.805091 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:51.805144 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 15:13:51.805230 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 15:13:51.805339 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":243,"spinlock_wait_cycles":5888,"thread_start_us":86,"threads_started":1}
I20260807 15:13:51.805539 10071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:51.800042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:33265 (local address 127.15.226.67:41063)
0807 15:13:51.800505 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:51.800519 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:51.800604 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:51.800907 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:51.800910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:51.800956 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:51.801197 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:51.801203 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.802301 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.802304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:51.803994 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:51.804007 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.804917 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.804921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:51.805104 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:51.805147 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 15:13:51.805230 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 15:13:51.805339 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":346,"spinlock_wait_cycles":34816,"thread_start_us":76,"threads_started":1}
I20260807 15:13:51.806385 9759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b03e5ff9c6b4c94a1e9f7bee9d5c2ba" candidate_uuid: "544dd2b84c8e4efbb48d05b4be836234" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59cb48711aa45fc8e1b4bafb3ec8d36" is_pre_election: true
I20260807 15:13:51.806443 9565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b03e5ff9c6b4c94a1e9f7bee9d5c2ba" candidate_uuid: "544dd2b84c8e4efbb48d05b4be836234" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b9da635d5b645f384c8068d821b0881" is_pre_election: true
I20260807 15:13:51.806771 9759 raft_consensus.cc:2454] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P c59cb48711aa45fc8e1b4bafb3ec8d36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 544dd2b84c8e4efbb48d05b4be836234 in term 0.
I20260807 15:13:51.806813 9565 raft_consensus.cc:2454] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 0b9da635d5b645f384c8068d821b0881 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 544dd2b84c8e4efbb48d05b4be836234 in term 0.
I20260807 15:13:51.807428 9868 leader_election.cc:302] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0b9da635d5b645f384c8068d821b0881, 544dd2b84c8e4efbb48d05b4be836234, c59cb48711aa45fc8e1b4bafb3ec8d36; no voters:
I20260807 15:13:51.807766 10066 raft_consensus.cc:2781] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:13:51.807904 10066 raft_consensus.cc:482] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:13:51.807936 10066 raft_consensus.cc:3037] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:51.809154 10066 raft_consensus.cc:504] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b9da635d5b645f384c8068d821b0881" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 33265 } } peers { permanent_uuid: "544dd2b84c8e4efbb48d05b4be836234" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46401 } } peers { permanent_uuid: "c59cb48711aa45fc8e1b4bafb3ec8d36" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 38551 } }
I20260807 15:13:51.809592 10066 leader_election.cc:288] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234 [CANDIDATE]: Term 1 election: Requested vote from peers 0b9da635d5b645f384c8068d821b0881 (127.15.226.65:33265), c59cb48711aa45fc8e1b4bafb3ec8d36 (127.15.226.66:38551)
I20260807 15:13:51.809990 9565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b03e5ff9c6b4c94a1e9f7bee9d5c2ba" candidate_uuid: "544dd2b84c8e4efbb48d05b4be836234" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b9da635d5b645f384c8068d821b0881"
I20260807 15:13:51.810109 9759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b03e5ff9c6b4c94a1e9f7bee9d5c2ba" candidate_uuid: "544dd2b84c8e4efbb48d05b4be836234" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c59cb48711aa45fc8e1b4bafb3ec8d36"
I20260807 15:13:51.810201 9565 raft_consensus.cc:3037] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 0b9da635d5b645f384c8068d821b0881 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:51.810263 9759 raft_consensus.cc:3037] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P c59cb48711aa45fc8e1b4bafb3ec8d36 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:51.811708 9565 raft_consensus.cc:2454] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 0b9da635d5b645f384c8068d821b0881 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 544dd2b84c8e4efbb48d05b4be836234 in term 1.
I20260807 15:13:51.811717 9759 raft_consensus.cc:2454] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P c59cb48711aa45fc8e1b4bafb3ec8d36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 544dd2b84c8e4efbb48d05b4be836234 in term 1.
I20260807 15:13:51.812110 9865 leader_election.cc:302] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0b9da635d5b645f384c8068d821b0881, 544dd2b84c8e4efbb48d05b4be836234, c59cb48711aa45fc8e1b4bafb3ec8d36; no voters:
I20260807 15:13:51.812354 10066 raft_consensus.cc:2781] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:13:51.812593 10066 raft_consensus.cc:686] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234 [term 1 LEADER]: Becoming Leader. State: Replica: 544dd2b84c8e4efbb48d05b4be836234, State: Running, Role: LEADER
I20260807 15:13:51.812963 10066 consensus_queue.cc:237] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b9da635d5b645f384c8068d821b0881" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 33265 } } peers { permanent_uuid: "544dd2b84c8e4efbb48d05b4be836234" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46401 } } peers { permanent_uuid: "c59cb48711aa45fc8e1b4bafb3ec8d36" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 38551 } }
I20260807 15:13:51.816558 9348 catalog_manager.cc:5495] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234 reported cstate change: term changed from 0 to 1, leader changed from <none> to 544dd2b84c8e4efbb48d05b4be836234 (127.15.226.67). New cstate: current_term: 1 leader_uuid: "544dd2b84c8e4efbb48d05b4be836234" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b9da635d5b645f384c8068d821b0881" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 33265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "544dd2b84c8e4efbb48d05b4be836234" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 46401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c59cb48711aa45fc8e1b4bafb3ec8d36" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 38551 } health_report { overall_health: UNKNOWN } } }
I20260807 15:13:51.891728 10081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:51.887476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57816 (local address 127.15.226.67:46401)
0807 15:13:51.887745 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:51.887750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:51.887765 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:13:51.887956 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:51.887959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:51.888022 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:51.888156 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:13:51.888446 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.888994 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.890001 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:51.890245 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:51.890375 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:51.890764 (+ 389us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:13:51.891288 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:13:51.891445 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:51.891484 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 15:13:51.891543 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":67,"threads_started":1}
I20260807 15:13:51.898231 9759 raft_consensus.cc:1261] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P c59cb48711aa45fc8e1b4bafb3ec8d36 [term 1 FOLLOWER]: Refusing update from remote peer 544dd2b84c8e4efbb48d05b4be836234: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:13:51.898231 9565 raft_consensus.cc:1261] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 0b9da635d5b645f384c8068d821b0881 [term 1 FOLLOWER]: Refusing update from remote peer 544dd2b84c8e4efbb48d05b4be836234: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:13:51.899040 10066 consensus_queue.cc:1035] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c59cb48711aa45fc8e1b4bafb3ec8d36" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 38551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:13:51.899302 10073 consensus_queue.cc:1035] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b9da635d5b645f384c8068d821b0881" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 33265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:13:51.910576 10087 mvcc.cc:205] Tried to move back new op lower bound from 7315929628246564864 to 7315929627908636672. Current Snapshot: MvccSnapshot[applied={T|T < 7315929628246564864 or (T in {7315929628246564864})}]
I20260807 15:13:51.917486 9350 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:13:51.917668 9350 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:13:51.920604 9350 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2cea7f749c26495fb51984a246c156b1: Sending DeleteTablet for 3 replicas of tablet 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba
I20260807 15:13:51.921657 9532 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:13:51 UTC) from {username='slave'} at 127.0.0.1:51186
I20260807 15:13:51.921816 9930 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:13:51 UTC) from {username='slave'} at 127.0.0.1:57812
I20260807 15:13:51.922531 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 9432
I20260807 15:13:51.922940 9727 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:13:51 UTC) from {username='slave'} at 127.0.0.1:51700
I20260807 15:13:51.928169 10104 tablet_replica.cc:330] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P c59cb48711aa45fc8e1b4bafb3ec8d36: stopping tablet replica
I20260807 15:13:51.928653 10104 raft_consensus.cc:2229] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P c59cb48711aa45fc8e1b4bafb3ec8d36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:13:51.928941 10103 tablet_replica.cc:330] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234: stopping tablet replica
I20260807 15:13:51.929126 10104 raft_consensus.cc:2258] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P c59cb48711aa45fc8e1b4bafb3ec8d36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:13:51.929426 10103 raft_consensus.cc:2229] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:13:51.929895 10103 raft_consensus.cc:2258] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:13:51.931883 10103 ts_tablet_manager.cc:1905] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:13:51.931885 10104 ts_tablet_manager.cc:1905] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P c59cb48711aa45fc8e1b4bafb3ec8d36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:13:51.935213 10103 ts_tablet_manager.cc:1918] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:13:51.935416 10103 log.cc:1196] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/8b03e5ff9c6b4c94a1e9f7bee9d5c2ba
I20260807 15:13:51.935869 10103 ts_tablet_manager.cc:1939] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P 544dd2b84c8e4efbb48d05b4be836234: Deleting consensus metadata
I20260807 15:13:51.935832 10104 ts_tablet_manager.cc:1918] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P c59cb48711aa45fc8e1b4bafb3ec8d36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:13:51.935958 10104 log.cc:1196] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P c59cb48711aa45fc8e1b4bafb3ec8d36: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/8b03e5ff9c6b4c94a1e9f7bee9d5c2ba
I20260807 15:13:51.936381 10104 ts_tablet_manager.cc:1939] T 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba P c59cb48711aa45fc8e1b4bafb3ec8d36: Deleting consensus metadata
I20260807 15:13:51.937592 9330 catalog_manager.cc:4841] TS c59cb48711aa45fc8e1b4bafb3ec8d36 (127.15.226.66:38551): tablet 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba (table test-table [id=74b5226284224ce69d33636680cc05a3]) successfully deleted
I20260807 15:13:51.938050 9331 catalog_manager.cc:4841] TS 544dd2b84c8e4efbb48d05b4be836234 (127.15.226.67:46401): tablet 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba (table test-table [id=74b5226284224ce69d33636680cc05a3]) successfully deleted
W20260807 15:13:51.944573 9327 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.226.65:33265 (error 108)
I20260807 15:13:51.944792 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 9636
I20260807 15:13:51.947796 10106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:51.945639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:33265 (local address 127.0.0.1:51190)
0807 15:13:51.947498 (+ 1859us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:51.947613 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.65:33265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1750,"thread_start_us":1463,"threads_started":1}
W20260807 15:13:51.947988 9327 catalog_manager.cc:4584] TS 0b9da635d5b645f384c8068d821b0881 (127.15.226.65:33265): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8b03e5ff9c6b4c94a1e9f7bee9d5c2ba: Network error: Client connection negotiation failed: client connection to 127.15.226.65:33265: connect: Connection refused (error 111)
I20260807 15:13:51.958878 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 9844
I20260807 15:13:51.979489 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 9308
2026-08-07T15:13:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1623 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:13:52 dist-test-slave-3kk6 krb5kdc[10111](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:13:52 dist-test-slave-3kk6 krb5kdc[10111](info): set up 2 sockets
Aug 07 15:13:52 dist-test-slave-3kk6 krb5kdc[10111](info): commencing operation
krb5kdc: starting...
W20260807 15:13:53.224691 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:13:53 dist-test-slave-3kk6 krb5kdc[10111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:13:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:13:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:13:53.362310 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:45707
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:35267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:45707
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:13:53.481287 10131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:53.481647 10131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:53.481729 10131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:53.486251 10131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:13:53.486403 10131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:53.486446 10131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:53.486490 10131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:13:53.486523 10131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:13:53.492363 10131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:35267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:45707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:45707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:53.493655 10131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:53.494851 10131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:53.504200 10140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:53.504202 10139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:53.504220 10142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:53.504609 10131 server_base.cc:1034] running on GCE node
I20260807 15:13:53.505232 10131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:53.506476 10131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:53.507663 10131 hybrid_clock.cc:648] HybridClock initialized: now 1786115633507643 us; error 46 us; skew 500 ppm
Aug 07 15:13:53 dist-test-slave-3kk6 krb5kdc[10111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115633, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:53.511258 10131 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:13:53.512378 10131 webserver.cc:460] Webserver started at http://127.15.226.126:38049/ using document root <none> and password file <none>
I20260807 15:13:53.513034 10131 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:53.513135 10131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:53.513371 10131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:13:53.515695 10131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "a261d8f79003473b9d5f82cbde752e01"
format_stamp: "Formatted at 2026-08-07 15:13:53 on dist-test-slave-3kk6"
server_key: "e076b2173e5b80a758af4ff18595c81a"
server_key_iv: "af6c348b5c1039826278f51905e349c5"
server_key_version: "encryptionkey@0"
I20260807 15:13:53.516351 10131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "a261d8f79003473b9d5f82cbde752e01"
format_stamp: "Formatted at 2026-08-07 15:13:53 on dist-test-slave-3kk6"
server_key: "e076b2173e5b80a758af4ff18595c81a"
server_key_iv: "af6c348b5c1039826278f51905e349c5"
server_key_version: "encryptionkey@0"
I20260807 15:13:53.520581 10131 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 15:13:53.523463 10153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:53.524909 10131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:13:53.525091 10131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "a261d8f79003473b9d5f82cbde752e01"
format_stamp: "Formatted at 2026-08-07 15:13:53 on dist-test-slave-3kk6"
server_key: "e076b2173e5b80a758af4ff18595c81a"
server_key_iv: "af6c348b5c1039826278f51905e349c5"
server_key_version: "encryptionkey@0"
I20260807 15:13:53.525277 10131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:53.549309 10131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:53.552819 10131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:53.553066 10131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:53.561635 10131 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:45707
I20260807 15:13:53.561656 10231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:45707 every 8 connection(s)
I20260807 15:13:53.562891 10131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:13:53.565977 10232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:53.571440 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 10131
I20260807 15:13:53.571624 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:13:53.571972 16265 external_mini_cluster.cc:1428] Setting key ca5c983d1471aa8d728565dbafbfe230
I20260807 15:13:53.572551 10232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a261d8f79003473b9d5f82cbde752e01: Bootstrap starting.
I20260807 15:13:53.575500 10232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a261d8f79003473b9d5f82cbde752e01: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:53.576623 10232 log.cc:824] T 00000000000000000000000000000000 P a261d8f79003473b9d5f82cbde752e01: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:53.579144 10232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a261d8f79003473b9d5f82cbde752e01: No bootstrap required, opened a new log
Aug 07 15:13:53 dist-test-slave-3kk6 krb5kdc[10111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:13:53.584477 10232 raft_consensus.cc:348] T 00000000000000000000000000000000 P a261d8f79003473b9d5f82cbde752e01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a261d8f79003473b9d5f82cbde752e01" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45707 } }
I20260807 15:13:53.584765 10232 raft_consensus.cc:374] T 00000000000000000000000000000000 P a261d8f79003473b9d5f82cbde752e01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:53.584802 10232 raft_consensus.cc:729] T 00000000000000000000000000000000 P a261d8f79003473b9d5f82cbde752e01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a261d8f79003473b9d5f82cbde752e01, State: Initialized, Role: FOLLOWER
I20260807 15:13:53.585350 10232 consensus_queue.cc:260] T 00000000000000000000000000000000 P a261d8f79003473b9d5f82cbde752e01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a261d8f79003473b9d5f82cbde752e01" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45707 } }
I20260807 15:13:53.585518 10232 raft_consensus.cc:388] T 00000000000000000000000000000000 P a261d8f79003473b9d5f82cbde752e01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:13:53.585615 10232 raft_consensus.cc:482] T 00000000000000000000000000000000 P a261d8f79003473b9d5f82cbde752e01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:13:53.585744 10232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a261d8f79003473b9d5f82cbde752e01 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:53.586933 10232 raft_consensus.cc:504] T 00000000000000000000000000000000 P a261d8f79003473b9d5f82cbde752e01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a261d8f79003473b9d5f82cbde752e01" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45707 } }
I20260807 15:13:53.587333 10232 leader_election.cc:302] T 00000000000000000000000000000000 P a261d8f79003473b9d5f82cbde752e01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a261d8f79003473b9d5f82cbde752e01; no voters:
I20260807 15:13:53.587759 10232 leader_election.cc:288] T 00000000000000000000000000000000 P a261d8f79003473b9d5f82cbde752e01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:13:53.587922 10239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a261d8f79003473b9d5f82cbde752e01 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:13:53.588151 10239 raft_consensus.cc:686] T 00000000000000000000000000000000 P a261d8f79003473b9d5f82cbde752e01 [term 1 LEADER]: Becoming Leader. State: Replica: a261d8f79003473b9d5f82cbde752e01, State: Running, Role: LEADER
I20260807 15:13:53.588497 10239 consensus_queue.cc:237] T 00000000000000000000000000000000 P a261d8f79003473b9d5f82cbde752e01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a261d8f79003473b9d5f82cbde752e01" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45707 } }
I20260807 15:13:53.588981 10236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:53.573597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57448 (local address 127.15.226.126:45707)
0807 15:13:53.574177 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:53.574194 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:53.574248 (+ 54us) server_negotiation.cc:407] Connection header received
0807 15:13:53.575065 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:53.575086 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:53.575460 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:53.575841 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:53.576907 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:53.577949 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:53.579185 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:53.579737 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:53.579879 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:53.583178 (+ 3299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:53.583185 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:53.583203 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:53.583237 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:53.585234 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:53.585866 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:53.585872 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:53.585877 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:53.586053 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:53.586413 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:53.586420 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:53.586423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:53.586972 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:53.587199 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:53.587518 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 15:13:53.587869 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":412,"thread_start_us":240,"threads_started":1}
I20260807 15:13:53.589257 10232 sys_catalog.cc:564] T 00000000000000000000000000000000 P a261d8f79003473b9d5f82cbde752e01 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:13:53.589829 10239 sys_catalog.cc:455] T 00000000000000000000000000000000 P a261d8f79003473b9d5f82cbde752e01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a261d8f79003473b9d5f82cbde752e01. Latest consensus state: current_term: 1 leader_uuid: "a261d8f79003473b9d5f82cbde752e01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a261d8f79003473b9d5f82cbde752e01" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45707 } } }
I20260807 15:13:53.589954 10239 sys_catalog.cc:458] T 00000000000000000000000000000000 P a261d8f79003473b9d5f82cbde752e01 [sys.catalog]: This master's current role is: LEADER
I20260807 15:13:53.590437 10240 sys_catalog.cc:455] T 00000000000000000000000000000000 P a261d8f79003473b9d5f82cbde752e01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a261d8f79003473b9d5f82cbde752e01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a261d8f79003473b9d5f82cbde752e01" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45707 } } }
I20260807 15:13:53.590626 10240 sys_catalog.cc:458] T 00000000000000000000000000000000 P a261d8f79003473b9d5f82cbde752e01 [sys.catalog]: This master's current role is: LEADER
I20260807 15:13:53.591292 10246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:13:53.595768 10246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:13:53.602468 10246 catalog_manager.cc:1347] Generated new cluster ID: 8a211c6d254247ea8bf78d2705a2e61f
I20260807 15:13:53.602581 10246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:13:53.625780 10246 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:13:53.626995 10246 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:13:53.636160 10246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a261d8f79003473b9d5f82cbde752e01: Generated new TSK 0
I20260807 15:13:53.637032 10246 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.226.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
I20260807 15:13:53.674059 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:45707
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:35267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:13:53.798980 10274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:53.799293 10274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:53.799342 10274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:53.803706 10274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:53.803793 10274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:53.803859 10274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:13:53.809880 10274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:35267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:45707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:53.811213 10274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:53.812469 10274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:53.822880 10283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:53.822926 10287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:53.822938 10281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:53.823280 10274 server_base.cc:1034] running on GCE node
I20260807 15:13:53.823735 10274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:53.824437 10274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:53.825641 10274 hybrid_clock.cc:648] HybridClock initialized: now 1786115633825626 us; error 36 us; skew 500 ppm
Aug 07 15:13:53 dist-test-slave-3kk6 krb5kdc[10111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115633, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:53.828753 10274 init.cc:377] Logged in from keytab as kudu/127.15.226.65@KRBTEST.COM (short username kudu)
I20260807 15:13:53.829878 10274 webserver.cc:460] Webserver started at http://127.15.226.65:44637/ using document root <none> and password file <none>
I20260807 15:13:53.830578 10274 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:53.830699 10274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:53.830932 10274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:13:53.833163 10274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "f903abe5420a4ecf958bf796301a00d1"
format_stamp: "Formatted at 2026-08-07 15:13:53 on dist-test-slave-3kk6"
server_key: "6d75093a4fe59f2516c16b8e9c2764d8"
server_key_iv: "d2d8af8c2ab42199131f7240ff146e6a"
server_key_version: "encryptionkey@0"
I20260807 15:13:53.833791 10274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "f903abe5420a4ecf958bf796301a00d1"
format_stamp: "Formatted at 2026-08-07 15:13:53 on dist-test-slave-3kk6"
server_key: "6d75093a4fe59f2516c16b8e9c2764d8"
server_key_iv: "d2d8af8c2ab42199131f7240ff146e6a"
server_key_version: "encryptionkey@0"
I20260807 15:13:53.838167 10274 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 15:13:53.840982 10300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:53.842281 10274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:13:53.842445 10274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "f903abe5420a4ecf958bf796301a00d1"
format_stamp: "Formatted at 2026-08-07 15:13:53 on dist-test-slave-3kk6"
server_key: "6d75093a4fe59f2516c16b8e9c2764d8"
server_key_iv: "d2d8af8c2ab42199131f7240ff146e6a"
server_key_version: "encryptionkey@0"
I20260807 15:13:53.842583 10274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:53.864775 10274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:53.868587 10274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:53.868830 10274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:53.869554 10274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:53.870507 10274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:53.870584 10274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:53.870684 10274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:53.870743 10274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:53.880981 10274 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:42183
I20260807 15:13:53.881000 10460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:42183 every 8 connection(s)
I20260807 15:13:53.882028 10274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:13:53.883241 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 10274
I20260807 15:13:53.883363 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:13:53.883695 16265 external_mini_cluster.cc:1428] Setting key 475f231065cfb50f3ceb41a4b60d4ef2
Aug 07 15:13:53 dist-test-slave-3kk6 krb5kdc[10111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115633, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.226.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.66@KRBTEST.COM" created.
I20260807 15:13:53.899839 10236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:53.884160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:47279 (local address 127.15.226.126:45707)
0807 15:13:53.884317 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:53.884322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:53.885247 (+ 925us) server_negotiation.cc:407] Connection header received
0807 15:13:53.886769 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:53.886776 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:53.886846 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:53.886949 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:53.888823 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:53.889457 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:53.891638 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:53.891920 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:53.892025 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:53.895366 (+ 3341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:53.895369 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:53.895371 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:53.895401 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:53.897079 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:53.897756 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:53.897760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:53.897761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:53.897855 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:53.898303 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:53.898307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:53.898308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:53.898528 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:53.898648 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:53.899492 (+ 844us) server_negotiation.cc:299] Negotiation successful
0807 15:13:53.899656 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 15:13:53.900933 10463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:53.884538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:45707 (local address 127.15.226.65:47279)
0807 15:13:53.885079 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:53.885141 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:53.886508 (+ 1367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:53.887160 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:53.887169 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:53.887774 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:53.888616 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:53.888645 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:53.889609 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:53.889616 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:53.891493 (+ 1877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:53.891509 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:53.892128 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:53.892135 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:53.892406 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:53.893181 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:13:53.893207 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:53.895133 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:53.897245 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:53.897253 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:53.897274 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:53.897613 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:53.897992 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:53.897995 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:53.897997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:53.898187 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:53.898679 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:53.898691 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:53.899137 (+ 446us) client_negotiation.cc:769] Sending connection context
0807 15:13:53.899442 (+ 305us) client_negotiation.cc:240] Negotiation successful
0807 15:13:53.899858 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":147,"threads_started":1}
I20260807 15:13:53.902556 10461 heartbeater.cc:347] Connected to a master server at 127.15.226.126:45707
I20260807 15:13:53.902910 10461 heartbeater.cc:464] Registering TS with master...
I20260807 15:13:53.903642 10461 heartbeater.cc:511] Master 127.15.226.126:45707 requested a full tablet report, sending...
I20260807 15:13:53.905736 10174 ts_manager.cc:194] Registered new tserver with Master: f903abe5420a4ecf958bf796301a00d1 (127.15.226.65:42183)
I20260807 15:13:53.907646 10174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.65@KRBTEST.COM'} at 127.15.226.65:47279
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
I20260807 15:13:53.921554 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:45707
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:35267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:13:54.051549 10466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:54.051932 10466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:54.052023 10466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:54.056499 10466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:54.056640 10466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:54.056748 10466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:13:54.062455 10466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:35267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:45707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:54.063701 10466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:54.064821 10466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:54.076174 10474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:54.076176 10477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:54.076187 10475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:54.076683 10466 server_base.cc:1034] running on GCE node
I20260807 15:13:54.077164 10466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:54.077929 10466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:54.079164 10466 hybrid_clock.cc:648] HybridClock initialized: now 1786115634079151 us; error 36 us; skew 500 ppm
Aug 07 15:13:54 dist-test-slave-3kk6 krb5kdc[10111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115634, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:54.082484 10466 init.cc:377] Logged in from keytab as kudu/127.15.226.66@KRBTEST.COM (short username kudu)
I20260807 15:13:54.083617 10466 webserver.cc:460] Webserver started at http://127.15.226.66:39501/ using document root <none> and password file <none>
I20260807 15:13:54.084257 10466 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:54.084368 10466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:54.084625 10466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:13:54.087011 10466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "876e7e0c99be4486b78492af922ba298"
format_stamp: "Formatted at 2026-08-07 15:13:54 on dist-test-slave-3kk6"
server_key: "f1c823cd53e9210ecb15e39e71690a04"
server_key_iv: "0d888a60408879b776339e731c747bf9"
server_key_version: "encryptionkey@0"
I20260807 15:13:54.087673 10466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "876e7e0c99be4486b78492af922ba298"
format_stamp: "Formatted at 2026-08-07 15:13:54 on dist-test-slave-3kk6"
server_key: "f1c823cd53e9210ecb15e39e71690a04"
server_key_iv: "0d888a60408879b776339e731c747bf9"
server_key_version: "encryptionkey@0"
I20260807 15:13:54.092136 10466 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:13:54.095155 10487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:54.097107 10466 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:13:54.097282 10466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "876e7e0c99be4486b78492af922ba298"
format_stamp: "Formatted at 2026-08-07 15:13:54 on dist-test-slave-3kk6"
server_key: "f1c823cd53e9210ecb15e39e71690a04"
server_key_iv: "0d888a60408879b776339e731c747bf9"
server_key_version: "encryptionkey@0"
I20260807 15:13:54.097460 10466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:54.113512 10466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:54.116765 10466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:54.116945 10466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:54.117547 10466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:54.118459 10466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:54.118510 10466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:54.118553 10466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:54.118569 10466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:54.128719 10466 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:37611
I20260807 15:13:54.128741 10646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:37611 every 8 connection(s)
I20260807 15:13:54.130084 10466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:13:54.130791 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 10466
I20260807 15:13:54.130946 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:13:54.131283 16265 external_mini_cluster.cc:1428] Setting key dbe209e779c30b24e13fc9b45b43202e
Aug 07 15:13:54 dist-test-slave-3kk6 krb5kdc[10111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115634, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.226.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.67@KRBTEST.COM" created.
I20260807 15:13:54.145898 10236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:54.132354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:53207 (local address 127.15.226.126:45707)
0807 15:13:54.132513 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:54.132517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:54.133673 (+ 1156us) server_negotiation.cc:407] Connection header received
0807 15:13:54.135197 (+ 1524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:54.135201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:54.135261 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:54.135353 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:54.136908 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.137464 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.138875 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.139114 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.139229 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:54.141969 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:54.141976 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:54.141980 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:54.142015 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:54.143634 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:54.144207 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:54.144211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:54.144213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:54.144302 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:54.144676 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:54.144680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:54.144682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:54.144885 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:54.145000 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:54.145572 (+ 572us) server_negotiation.cc:299] Negotiation successful
0807 15:13:54.145723 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 15:13:54.147011 10651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:54.132876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:45707 (local address 127.15.226.66:53207)
0807 15:13:54.133476 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:54.133538 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:54.134965 (+ 1427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:54.135553 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:54.135560 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:54.136055 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:54.136742 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:54.136761 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.137614 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.137617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:54.138741 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:54.138752 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.139302 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.139306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:54.139493 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:54.140170 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:13:54.140188 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:54.141773 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:54.143769 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:54.143775 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:54.143793 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:54.144086 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:54.144402 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:54.144405 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:54.144407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:54.144570 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:54.145001 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:54.145009 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:54.145311 (+ 302us) client_negotiation.cc:769] Sending connection context
0807 15:13:54.145524 (+ 213us) client_negotiation.cc:240] Negotiation successful
0807 15:13:54.145901 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":166,"threads_started":1}
I20260807 15:13:54.148387 10649 heartbeater.cc:347] Connected to a master server at 127.15.226.126:45707
I20260807 15:13:54.148680 10649 heartbeater.cc:464] Registering TS with master...
I20260807 15:13:54.149248 10649 heartbeater.cc:511] Master 127.15.226.126:45707 requested a full tablet report, sending...
I20260807 15:13:54.150506 10174 ts_manager.cc:194] Registered new tserver with Master: 876e7e0c99be4486b78492af922ba298 (127.15.226.66:37611)
I20260807 15:13:54.151453 10174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.66@KRBTEST.COM'} at 127.15.226.66:53207
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
I20260807 15:13:54.166307 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:45707
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:35267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:13:54.288014 10657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:54.288322 10657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:54.288368 10657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:54.292897 10657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:54.292987 10657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:54.293051 10657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:13:54.298841 10657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:35267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:45707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:54.300092 10657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:54.301470 10657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:54.312439 10665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:54.312472 10663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:54.312485 10662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:54.312808 10657 server_base.cc:1034] running on GCE node
I20260807 15:13:54.313381 10657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:54.314152 10657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:54.315356 10657 hybrid_clock.cc:648] HybridClock initialized: now 1786115634315322 us; error 49 us; skew 500 ppm
Aug 07 15:13:54 dist-test-slave-3kk6 krb5kdc[10111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115634, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:54.318449 10657 init.cc:377] Logged in from keytab as kudu/127.15.226.67@KRBTEST.COM (short username kudu)
I20260807 15:13:54.319473 10657 webserver.cc:460] Webserver started at http://127.15.226.67:45331/ using document root <none> and password file <none>
I20260807 15:13:54.320122 10657 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:54.320225 10657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:54.320477 10657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:13:54.322808 10657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "cd1a2189f0a24bf080b5eb9a46a63ba6"
format_stamp: "Formatted at 2026-08-07 15:13:54 on dist-test-slave-3kk6"
server_key: "f9faa9e939779b2b929600e6649ff9bf"
server_key_iv: "3f317ddaf9da8a00c978f10b33d31b91"
server_key_version: "encryptionkey@0"
I20260807 15:13:54.323457 10657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "cd1a2189f0a24bf080b5eb9a46a63ba6"
format_stamp: "Formatted at 2026-08-07 15:13:54 on dist-test-slave-3kk6"
server_key: "f9faa9e939779b2b929600e6649ff9bf"
server_key_iv: "3f317ddaf9da8a00c978f10b33d31b91"
server_key_version: "encryptionkey@0"
I20260807 15:13:54.327595 10657 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:13:54.330401 10675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:54.331879 10657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 15:13:54.332046 10657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "cd1a2189f0a24bf080b5eb9a46a63ba6"
format_stamp: "Formatted at 2026-08-07 15:13:54 on dist-test-slave-3kk6"
server_key: "f9faa9e939779b2b929600e6649ff9bf"
server_key_iv: "3f317ddaf9da8a00c978f10b33d31b91"
server_key_version: "encryptionkey@0"
I20260807 15:13:54.332176 10657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:54.356117 10657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:54.359973 10657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:54.360224 10657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:54.360932 10657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:54.361896 10657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:54.361980 10657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:54.362053 10657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:54.362102 10657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:54.371843 10657 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:42583
I20260807 15:13:54.371908 10843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:42583 every 8 connection(s)
I20260807 15:13:54.372843 10657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:13:54.375334 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 10657
I20260807 15:13:54.375509 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:13:54.375873 16265 external_mini_cluster.cc:1428] Setting key d3d083c3135db101b8bc2acc4eb5d395
Aug 07 15:13:54 dist-test-slave-3kk6 krb5kdc[10111](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115634, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:13:54.388609 10236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:54.374843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:54095 (local address 127.15.226.126:45707)
0807 15:13:54.375008 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:54.375013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:54.376149 (+ 1136us) server_negotiation.cc:407] Connection header received
0807 15:13:54.377893 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:54.377896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:54.377961 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:54.378060 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:54.379789 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.380330 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.381764 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.381986 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.382069 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:54.384753 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:54.384756 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:54.384759 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:54.384786 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:54.386370 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:54.387040 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:54.387043 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:54.387044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:54.387132 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:54.387472 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:54.387474 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:54.387475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:54.387667 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:54.387746 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:54.388354 (+ 608us) server_negotiation.cc:299] Negotiation successful
0807 15:13:54.388476 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 15:13:54.389968 10848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:54.375301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:45707 (local address 127.15.226.67:54095)
0807 15:13:54.375941 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:54.376011 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:54.377660 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:54.378547 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:54.378557 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:54.379085 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:54.379620 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:54.379635 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.380480 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.380483 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:54.381640 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:54.381652 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.382135 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.382138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:54.382294 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:54.382979 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:13:54.382996 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:54.384578 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:54.386504 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:54.386509 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:54.386523 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:54.386920 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:54.387221 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:54.387223 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:54.387224 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:54.387376 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:54.387809 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:54.387815 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:54.388100 (+ 285us) client_negotiation.cc:769] Sending connection context
0807 15:13:54.388313 (+ 213us) client_negotiation.cc:240] Negotiation successful
0807 15:13:54.388813 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":197,"threads_started":1}
I20260807 15:13:54.391341 10844 heartbeater.cc:347] Connected to a master server at 127.15.226.126:45707
I20260807 15:13:54.391614 10844 heartbeater.cc:464] Registering TS with master...
I20260807 15:13:54.392154 10844 heartbeater.cc:511] Master 127.15.226.126:45707 requested a full tablet report, sending...
I20260807 15:13:54.393225 10174 ts_manager.cc:194] Registered new tserver with Master: cd1a2189f0a24bf080b5eb9a46a63ba6 (127.15.226.67:42583)
I20260807 15:13:54.393989 10174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.67@KRBTEST.COM'} at 127.15.226.67:54095
I20260807 15:13:54.402848 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:13:54.414202 10236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:54.405248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57454 (local address 127.15.226.126:45707)
0807 15:13:54.405433 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:54.405438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:54.405596 (+ 158us) server_negotiation.cc:407] Connection header received
0807 15:13:54.405772 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:54.405780 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:54.405825 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:54.405915 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:54.406744 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.407290 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.408253 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.408485 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.408614 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:54.410437 (+ 1823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:54.410443 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:54.410446 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:54.410484 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:54.412160 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:54.412732 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:54.412742 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:54.412744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:54.412826 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:54.413193 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:54.413196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:54.413198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:54.413431 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:54.413603 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:54.413907 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 15:13:54.414022 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 15:13:54.417500 10174 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:13:54.420822 10174 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:13:54.436796 10868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:54.429535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:42583 (local address 127.0.0.1:57922)
0807 15:13:54.430465 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:54.430523 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:54.430701 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:54.432305 (+ 1604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:54.432310 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:54.432329 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:54.432421 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:54.432427 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.433798 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.433801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:54.434835 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:54.434845 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.436038 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.436048 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:54.436199 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:54.436250 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 15:13:54.436475 (+ 225us) client_negotiation.cc:240] Negotiation successful
0807 15:13:54.436646 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":14,"thread_start_us":84,"threads_started":1}
I20260807 15:13:54.436837 10866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:54.429658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57922 (local address 127.15.226.67:42583)
0807 15:13:54.431836 (+ 2178us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:54.431850 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:54.431878 (+ 28us) server_negotiation.cc:407] Connection header received
0807 15:13:54.431971 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:54.431976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:54.432156 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:54.432312 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:54.432561 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.433649 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.434971 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.435804 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.435976 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:54.436173 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:54.436605 (+ 432us) server_negotiation.cc:299] Negotiation successful
0807 15:13:54.436707 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1986,"thread_start_us":121,"threads_started":1}
I20260807 15:13:54.437706 10872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:54.430412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39130 (local address 127.15.226.66:37611)
0807 15:13:54.431015 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:54.431028 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:54.431055 (+ 27us) server_negotiation.cc:407] Connection header received
0807 15:13:54.431134 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:54.431140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:54.431357 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:54.431547 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:54.432084 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.433303 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.435261 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.436118 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.436234 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:54.436369 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:54.436524 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 15:13:54.436629 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":123,"threads_started":1}
I20260807 15:13:54.438177 10862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:54.429641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:37611 (local address 127.0.0.1:39130)
0807 15:13:54.430678 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:54.430693 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:54.430793 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:54.431784 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:54.431790 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:54.431817 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:54.431951 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:54.431958 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.433452 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.433455 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:54.435120 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:54.435133 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.436243 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.436247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:54.436331 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:54.436358 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 15:13:54.436489 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 15:13:54.438050 (+ 1561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":921,"mutex_wait_us":131,"thread_start_us":122,"threads_started":1}
I20260807 15:13:54.438880 10863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:54.429935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:42183 (local address 127.0.0.1:33456)
0807 15:13:54.431197 (+ 1262us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:54.431213 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:54.431351 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:54.432495 (+ 1144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:54.432498 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:54.432519 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:54.432626 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:54.432631 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.433669 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.433672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:54.435126 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:54.435133 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.437760 (+ 2627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.437764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:54.437868 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:54.438455 (+ 587us) client_negotiation.cc:769] Sending connection context
0807 15:13:54.438565 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 15:13:54.438672 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1122,"mutex_wait_us":94,"thread_start_us":111,"threads_started":1}
I20260807 15:13:54.438895 10870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:54.430231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33456 (local address 127.15.226.65:42183)
0807 15:13:54.430553 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:54.430562 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:54.432025 (+ 1463us) server_negotiation.cc:407] Connection header received
0807 15:13:54.432107 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:54.432111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:54.432276 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:54.432398 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:54.432753 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.433530 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.436265 (+ 2735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.437619 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.437746 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:54.438059 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:54.438610 (+ 551us) server_negotiation.cc:299] Negotiation successful
0807 15:13:54.438754 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":128,"threads_started":1}
I20260807 15:13:54.440326 10749 tablet_service.cc:1467] Processing CreateTablet for tablet b79f6394250247ba918f863a0f8362c3 (DEFAULT_TABLE table=test-table [id=58b49a6059344a9a985a5b9bf98288f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:13:54.441387 10749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b79f6394250247ba918f863a0f8362c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:54.441748 10368 tablet_service.cc:1467] Processing CreateTablet for tablet b79f6394250247ba918f863a0f8362c3 (DEFAULT_TABLE table=test-table [id=58b49a6059344a9a985a5b9bf98288f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:13:54.442852 10368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b79f6394250247ba918f863a0f8362c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:54.442831 10556 tablet_service.cc:1467] Processing CreateTablet for tablet b79f6394250247ba918f863a0f8362c3 (DEFAULT_TABLE table=test-table [id=58b49a6059344a9a985a5b9bf98288f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:13:54.443840 10556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b79f6394250247ba918f863a0f8362c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:54.451152 10874 tablet_bootstrap.cc:492] T b79f6394250247ba918f863a0f8362c3 P cd1a2189f0a24bf080b5eb9a46a63ba6: Bootstrap starting.
I20260807 15:13:54.451157 10875 tablet_bootstrap.cc:492] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1: Bootstrap starting.
I20260807 15:13:54.451679 10876 tablet_bootstrap.cc:492] T b79f6394250247ba918f863a0f8362c3 P 876e7e0c99be4486b78492af922ba298: Bootstrap starting.
I20260807 15:13:54.453895 10874 tablet_bootstrap.cc:654] T b79f6394250247ba918f863a0f8362c3 P cd1a2189f0a24bf080b5eb9a46a63ba6: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:54.453919 10875 tablet_bootstrap.cc:654] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:54.455451 10874 log.cc:824] T b79f6394250247ba918f863a0f8362c3 P cd1a2189f0a24bf080b5eb9a46a63ba6: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:54.455638 10875 log.cc:824] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:54.456458 10876 tablet_bootstrap.cc:654] T b79f6394250247ba918f863a0f8362c3 P 876e7e0c99be4486b78492af922ba298: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:54.457523 10876 log.cc:824] T b79f6394250247ba918f863a0f8362c3 P 876e7e0c99be4486b78492af922ba298: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:54.457741 10874 tablet_bootstrap.cc:492] T b79f6394250247ba918f863a0f8362c3 P cd1a2189f0a24bf080b5eb9a46a63ba6: No bootstrap required, opened a new log
I20260807 15:13:54.457975 10874 ts_tablet_manager.cc:1397] T b79f6394250247ba918f863a0f8362c3 P cd1a2189f0a24bf080b5eb9a46a63ba6: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20260807 15:13:54.459057 10875 tablet_bootstrap.cc:492] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1: No bootstrap required, opened a new log
I20260807 15:13:54.459273 10875 ts_tablet_manager.cc:1397] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 15:13:54.459527 10876 tablet_bootstrap.cc:492] T b79f6394250247ba918f863a0f8362c3 P 876e7e0c99be4486b78492af922ba298: No bootstrap required, opened a new log
I20260807 15:13:54.459731 10876 ts_tablet_manager.cc:1397] T b79f6394250247ba918f863a0f8362c3 P 876e7e0c99be4486b78492af922ba298: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 15:13:54.462368 10874 raft_consensus.cc:348] T b79f6394250247ba918f863a0f8362c3 P cd1a2189f0a24bf080b5eb9a46a63ba6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd1a2189f0a24bf080b5eb9a46a63ba6" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 42583 } } peers { permanent_uuid: "876e7e0c99be4486b78492af922ba298" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 37611 } } peers { permanent_uuid: "f903abe5420a4ecf958bf796301a00d1" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42183 } }
I20260807 15:13:54.462692 10874 raft_consensus.cc:374] T b79f6394250247ba918f863a0f8362c3 P cd1a2189f0a24bf080b5eb9a46a63ba6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:54.462769 10874 raft_consensus.cc:729] T b79f6394250247ba918f863a0f8362c3 P cd1a2189f0a24bf080b5eb9a46a63ba6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd1a2189f0a24bf080b5eb9a46a63ba6, State: Initialized, Role: FOLLOWER
I20260807 15:13:54.463145 10876 raft_consensus.cc:348] T b79f6394250247ba918f863a0f8362c3 P 876e7e0c99be4486b78492af922ba298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd1a2189f0a24bf080b5eb9a46a63ba6" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 42583 } } peers { permanent_uuid: "876e7e0c99be4486b78492af922ba298" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 37611 } } peers { permanent_uuid: "f903abe5420a4ecf958bf796301a00d1" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42183 } }
I20260807 15:13:54.463268 10874 consensus_queue.cc:260] T b79f6394250247ba918f863a0f8362c3 P cd1a2189f0a24bf080b5eb9a46a63ba6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd1a2189f0a24bf080b5eb9a46a63ba6" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 42583 } } peers { permanent_uuid: "876e7e0c99be4486b78492af922ba298" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 37611 } } peers { permanent_uuid: "f903abe5420a4ecf958bf796301a00d1" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42183 } }
I20260807 15:13:54.463377 10876 raft_consensus.cc:374] T b79f6394250247ba918f863a0f8362c3 P 876e7e0c99be4486b78492af922ba298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:54.463441 10876 raft_consensus.cc:729] T b79f6394250247ba918f863a0f8362c3 P 876e7e0c99be4486b78492af922ba298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 876e7e0c99be4486b78492af922ba298, State: Initialized, Role: FOLLOWER
I20260807 15:13:54.463917 10876 consensus_queue.cc:260] T b79f6394250247ba918f863a0f8362c3 P 876e7e0c99be4486b78492af922ba298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd1a2189f0a24bf080b5eb9a46a63ba6" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 42583 } } peers { permanent_uuid: "876e7e0c99be4486b78492af922ba298" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 37611 } } peers { permanent_uuid: "f903abe5420a4ecf958bf796301a00d1" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42183 } }
I20260807 15:13:54.464074 10844 heartbeater.cc:503] Master 127.15.226.126:45707 was elected leader, sending a full tablet report...
I20260807 15:13:54.464398 10874 ts_tablet_manager.cc:1428] T b79f6394250247ba918f863a0f8362c3 P cd1a2189f0a24bf080b5eb9a46a63ba6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 15:13:54.464718 10649 heartbeater.cc:503] Master 127.15.226.126:45707 was elected leader, sending a full tablet report...
I20260807 15:13:54.465384 10875 raft_consensus.cc:348] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd1a2189f0a24bf080b5eb9a46a63ba6" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 42583 } } peers { permanent_uuid: "876e7e0c99be4486b78492af922ba298" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 37611 } } peers { permanent_uuid: "f903abe5420a4ecf958bf796301a00d1" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42183 } }
I20260807 15:13:54.465699 10875 raft_consensus.cc:374] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:54.465760 10875 raft_consensus.cc:729] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f903abe5420a4ecf958bf796301a00d1, State: Initialized, Role: FOLLOWER
I20260807 15:13:54.466172 10875 consensus_queue.cc:260] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd1a2189f0a24bf080b5eb9a46a63ba6" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 42583 } } peers { permanent_uuid: "876e7e0c99be4486b78492af922ba298" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 37611 } } peers { permanent_uuid: "f903abe5420a4ecf958bf796301a00d1" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42183 } }
I20260807 15:13:54.466794 10461 heartbeater.cc:503] Master 127.15.226.126:45707 was elected leader, sending a full tablet report...
I20260807 15:13:54.466866 10876 ts_tablet_manager.cc:1428] T b79f6394250247ba918f863a0f8362c3 P 876e7e0c99be4486b78492af922ba298: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 15:13:54.468094 10875 ts_tablet_manager.cc:1428] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1: Time spent starting tablet: real 0.009s user 0.001s sys 0.006s
I20260807 15:13:54.583839 10885 raft_consensus.cc:482] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:13:54.584147 10885 raft_consensus.cc:504] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd1a2189f0a24bf080b5eb9a46a63ba6" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 42583 } } peers { permanent_uuid: "876e7e0c99be4486b78492af922ba298" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 37611 } } peers { permanent_uuid: "f903abe5420a4ecf958bf796301a00d1" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42183 } }
I20260807 15:13:54.585845 10885 leader_election.cc:288] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd1a2189f0a24bf080b5eb9a46a63ba6 (127.15.226.67:42583), 876e7e0c99be4486b78492af922ba298 (127.15.226.66:37611)
I20260807 15:13:54.591248 10872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:54.586054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:45975 (local address 127.15.226.66:37611)
0807 15:13:54.586264 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:54.586269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:54.586920 (+ 651us) server_negotiation.cc:407] Connection header received
0807 15:13:54.587132 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:54.587138 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:54.587197 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:54.587290 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:54.587554 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.588238 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.589777 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.590448 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.590587 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:54.590643 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:54.591023 (+ 380us) server_negotiation.cc:299] Negotiation successful
0807 15:13:54.591105 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260807 15:13:54.591248 10866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:54.586096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:46731 (local address 127.15.226.67:42583)
0807 15:13:54.586312 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:54.586317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:54.586401 (+ 84us) server_negotiation.cc:407] Connection header received
0807 15:13:54.586703 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:54.586717 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:54.586777 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:54.586862 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:54.587213 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.588009 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.589773 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.590448 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.590586 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:54.590643 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:54.591023 (+ 380us) server_negotiation.cc:299] Negotiation successful
0807 15:13:54.591105 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260807 15:13:54.591321 10887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:54.585806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:37611 (local address 127.15.226.65:45975)
0807 15:13:54.586824 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:54.586841 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:54.587001 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:54.587325 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:54.587327 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:54.587344 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:54.587441 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:54.587445 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.588337 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.588340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:54.589610 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:54.589624 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.590594 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.590598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:54.590822 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:54.590883 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 15:13:54.590969 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 15:13:54.591116 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"mutex_wait_us":619,"thread_start_us":121,"threads_started":1}
I20260807 15:13:54.591670 10886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:54.585806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:42583 (local address 127.15.226.65:46731)
0807 15:13:54.586277 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:54.586295 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:54.586500 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:54.586888 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:54.586894 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:54.586921 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:54.587111 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:54.587118 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.588126 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.588129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:54.589610 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:54.589624 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.590594 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.590598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:54.590828 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:54.590877 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 15:13:54.590987 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 15:13:54.591552 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":562,"thread_start_us":163,"threads_started":1}
I20260807 15:13:54.592082 10588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b79f6394250247ba918f863a0f8362c3" candidate_uuid: "f903abe5420a4ecf958bf796301a00d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "876e7e0c99be4486b78492af922ba298" is_pre_election: true
I20260807 15:13:54.592344 10781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b79f6394250247ba918f863a0f8362c3" candidate_uuid: "f903abe5420a4ecf958bf796301a00d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd1a2189f0a24bf080b5eb9a46a63ba6" is_pre_election: true
I20260807 15:13:54.592442 10588 raft_consensus.cc:2454] T b79f6394250247ba918f863a0f8362c3 P 876e7e0c99be4486b78492af922ba298 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f903abe5420a4ecf958bf796301a00d1 in term 0.
I20260807 15:13:54.592674 10781 raft_consensus.cc:2454] T b79f6394250247ba918f863a0f8362c3 P cd1a2189f0a24bf080b5eb9a46a63ba6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f903abe5420a4ecf958bf796301a00d1 in term 0.
I20260807 15:13:54.593063 10301 leader_election.cc:302] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 876e7e0c99be4486b78492af922ba298, f903abe5420a4ecf958bf796301a00d1; no voters:
I20260807 15:13:54.593384 10885 raft_consensus.cc:2781] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:13:54.593495 10885 raft_consensus.cc:482] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:13:54.593557 10885 raft_consensus.cc:3037] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:54.594988 10885 raft_consensus.cc:504] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd1a2189f0a24bf080b5eb9a46a63ba6" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 42583 } } peers { permanent_uuid: "876e7e0c99be4486b78492af922ba298" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 37611 } } peers { permanent_uuid: "f903abe5420a4ecf958bf796301a00d1" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42183 } }
I20260807 15:13:54.595391 10885 leader_election.cc:288] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1 [CANDIDATE]: Term 1 election: Requested vote from peers cd1a2189f0a24bf080b5eb9a46a63ba6 (127.15.226.67:42583), 876e7e0c99be4486b78492af922ba298 (127.15.226.66:37611)
I20260807 15:13:54.595933 10588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b79f6394250247ba918f863a0f8362c3" candidate_uuid: "f903abe5420a4ecf958bf796301a00d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "876e7e0c99be4486b78492af922ba298"
I20260807 15:13:54.595950 10781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b79f6394250247ba918f863a0f8362c3" candidate_uuid: "f903abe5420a4ecf958bf796301a00d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd1a2189f0a24bf080b5eb9a46a63ba6"
I20260807 15:13:54.596100 10588 raft_consensus.cc:3037] T b79f6394250247ba918f863a0f8362c3 P 876e7e0c99be4486b78492af922ba298 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:54.596107 10781 raft_consensus.cc:3037] T b79f6394250247ba918f863a0f8362c3 P cd1a2189f0a24bf080b5eb9a46a63ba6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:54.597285 10588 raft_consensus.cc:2454] T b79f6394250247ba918f863a0f8362c3 P 876e7e0c99be4486b78492af922ba298 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f903abe5420a4ecf958bf796301a00d1 in term 1.
I20260807 15:13:54.597288 10781 raft_consensus.cc:2454] T b79f6394250247ba918f863a0f8362c3 P cd1a2189f0a24bf080b5eb9a46a63ba6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f903abe5420a4ecf958bf796301a00d1 in term 1.
I20260807 15:13:54.597669 10301 leader_election.cc:302] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 876e7e0c99be4486b78492af922ba298, cd1a2189f0a24bf080b5eb9a46a63ba6, f903abe5420a4ecf958bf796301a00d1; no voters:
I20260807 15:13:54.597918 10885 raft_consensus.cc:2781] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:13:54.598204 10885 raft_consensus.cc:686] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1 [term 1 LEADER]: Becoming Leader. State: Replica: f903abe5420a4ecf958bf796301a00d1, State: Running, Role: LEADER
I20260807 15:13:54.598577 10885 consensus_queue.cc:237] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd1a2189f0a24bf080b5eb9a46a63ba6" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 42583 } } peers { permanent_uuid: "876e7e0c99be4486b78492af922ba298" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 37611 } } peers { permanent_uuid: "f903abe5420a4ecf958bf796301a00d1" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42183 } }
I20260807 15:13:54.602217 10172 catalog_manager.cc:5495] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to f903abe5420a4ecf958bf796301a00d1 (127.15.226.65). New cstate: current_term: 1 leader_uuid: "f903abe5420a4ecf958bf796301a00d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd1a2189f0a24bf080b5eb9a46a63ba6" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 42583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "876e7e0c99be4486b78492af922ba298" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 37611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f903abe5420a4ecf958bf796301a00d1" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42183 } health_report { overall_health: HEALTHY } } }
W20260807 15:13:54.623950 10847 tablet.cc:2369] T b79f6394250247ba918f863a0f8362c3 P cd1a2189f0a24bf080b5eb9a46a63ba6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:13:54.631456 10650 tablet.cc:2369] T b79f6394250247ba918f863a0f8362c3 P 876e7e0c99be4486b78492af922ba298: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:13:54.633419 10870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:54.629486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33458 (local address 127.15.226.65:42183)
0807 15:13:54.629658 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:54.629662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:54.629679 (+ 17us) server_negotiation.cc:407] Connection header received
0807 15:13:54.629874 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:54.629877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:54.629935 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:54.630029 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:13:54.630294 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.630912 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.631857 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.632179 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.632270 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:54.632565 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:13:54.632982 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:13:54.633095 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:54.633188 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 15:13:54.633228 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260807 15:13:54.633455 10462 tablet.cc:2369] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:13:54.639812 10588 raft_consensus.cc:1261] T b79f6394250247ba918f863a0f8362c3 P 876e7e0c99be4486b78492af922ba298 [term 1 FOLLOWER]: Refusing update from remote peer f903abe5420a4ecf958bf796301a00d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:13:54.639827 10781 raft_consensus.cc:1261] T b79f6394250247ba918f863a0f8362c3 P cd1a2189f0a24bf080b5eb9a46a63ba6 [term 1 FOLLOWER]: Refusing update from remote peer f903abe5420a4ecf958bf796301a00d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:13:54.640542 10885 consensus_queue.cc:1035] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "876e7e0c99be4486b78492af922ba298" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 37611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:13:54.640798 10890 consensus_queue.cc:1035] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd1a2189f0a24bf080b5eb9a46a63ba6" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 42583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:13:54.648936 10895 mvcc.cc:205] Tried to move back new op lower bound from 7315929639476838400 to 7315929639318876160. Current Snapshot: MvccSnapshot[applied={T|T < 7315929639476838400}]
I20260807 15:13:54.658903 10172 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:13:54.659140 10172 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:13:54.662128 10172 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a261d8f79003473b9d5f82cbde752e01: Sending DeleteTablet for 3 replicas of tablet b79f6394250247ba918f863a0f8362c3
I20260807 15:13:54.663203 10556 tablet_service.cc:1514] Processing DeleteTablet for tablet b79f6394250247ba918f863a0f8362c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:13:54 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:39130
I20260807 15:13:54.663205 10368 tablet_service.cc:1514] Processing DeleteTablet for tablet b79f6394250247ba918f863a0f8362c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:13:54 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:33456
I20260807 15:13:54.663877 10907 tablet_replica.cc:330] T b79f6394250247ba918f863a0f8362c3 P f903abe5420a4ecf958bf796301a00d1: stopping tablet replica
I20260807 15:13:54.663870 10749 tablet_service.cc:1514] Processing DeleteTablet for tablet b79f6394250247ba918f863a0f8362c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:13:54 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:57922
I20260807 15:13:54.664232 10908 tablet_replica.cc:330] T b79f6394250247ba918f863a0f8362c3 P 876e7e0c99be4486b78492af922ba298: stopping tablet replica
I20260807 15:13:54.664354 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 10274
I20260807 15:13:54.664418 10909 tablet_replica.cc:330] T b79f6394250247ba918f863a0f8362c3 P cd1a2189f0a24bf080b5eb9a46a63ba6: stopping tablet replica
I20260807 15:13:54.670698 10908 raft_consensus.cc:2229] T b79f6394250247ba918f863a0f8362c3 P 876e7e0c99be4486b78492af922ba298 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:13:54.671416 10908 raft_consensus.cc:2258] T b79f6394250247ba918f863a0f8362c3 P 876e7e0c99be4486b78492af922ba298 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:13:54.675491 10909 raft_consensus.cc:2229] T b79f6394250247ba918f863a0f8362c3 P cd1a2189f0a24bf080b5eb9a46a63ba6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:13:54.675719 10908 ts_tablet_manager.cc:1905] T b79f6394250247ba918f863a0f8362c3 P 876e7e0c99be4486b78492af922ba298: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:13:54.676014 10909 raft_consensus.cc:2258] T b79f6394250247ba918f863a0f8362c3 P cd1a2189f0a24bf080b5eb9a46a63ba6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:13:54.677913 10909 ts_tablet_manager.cc:1905] T b79f6394250247ba918f863a0f8362c3 P cd1a2189f0a24bf080b5eb9a46a63ba6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:13:54.678922 10908 ts_tablet_manager.cc:1918] T b79f6394250247ba918f863a0f8362c3 P 876e7e0c99be4486b78492af922ba298: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:13:54.679051 10908 log.cc:1196] T b79f6394250247ba918f863a0f8362c3 P 876e7e0c99be4486b78492af922ba298: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/b79f6394250247ba918f863a0f8362c3
I20260807 15:13:54.679387 10908 ts_tablet_manager.cc:1939] T b79f6394250247ba918f863a0f8362c3 P 876e7e0c99be4486b78492af922ba298: Deleting consensus metadata
I20260807 15:13:54.680663 10154 catalog_manager.cc:4841] TS 876e7e0c99be4486b78492af922ba298 (127.15.226.66:37611): tablet b79f6394250247ba918f863a0f8362c3 (table test-table [id=58b49a6059344a9a985a5b9bf98288f5]) successfully deleted
I20260807 15:13:54.681727 10909 ts_tablet_manager.cc:1918] T b79f6394250247ba918f863a0f8362c3 P cd1a2189f0a24bf080b5eb9a46a63ba6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:13:54.681852 10909 log.cc:1196] T b79f6394250247ba918f863a0f8362c3 P cd1a2189f0a24bf080b5eb9a46a63ba6: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/b79f6394250247ba918f863a0f8362c3
I20260807 15:13:54.682247 10909 ts_tablet_manager.cc:1939] T b79f6394250247ba918f863a0f8362c3 P cd1a2189f0a24bf080b5eb9a46a63ba6: Deleting consensus metadata
I20260807 15:13:54.683807 10155 catalog_manager.cc:4841] TS cd1a2189f0a24bf080b5eb9a46a63ba6 (127.15.226.67:42583): tablet b79f6394250247ba918f863a0f8362c3 (table test-table [id=58b49a6059344a9a985a5b9bf98288f5]) successfully deleted
W20260807 15:13:54.684391 10157 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.226.65:42183 (error 108)
I20260807 15:13:54.684708 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 10466
I20260807 15:13:54.687724 10863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:54.687338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:42183 (local address 127.0.0.1:33464)
0807 15:13:54.687523 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:54.687616 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.65:42183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260807 15:13:54.687914 10157 catalog_manager.cc:4584] TS f903abe5420a4ecf958bf796301a00d1 (127.15.226.65:42183): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b79f6394250247ba918f863a0f8362c3: Network error: Client connection negotiation failed: client connection to 127.15.226.65:42183: connect: Connection refused (error 111)
I20260807 15:13:54.698539 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 10657
I20260807 15:13:54.718935 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 10131
2026-08-07T15:13:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2740 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T15:13:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:13:54Z Disabled control of system clock
I20260807 15:13:54.760061 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:43275
--webserver_interface=127.15.226.126
--webserver_port=0
--builtin_ntp_servers=127.15.226.84:35309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:43275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:13:54.882828 10917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:54.883175 10917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:54.883215 10917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:54.887646 10917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:13:54.887744 10917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:54.887760 10917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:54.887779 10917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:13:54.887796 10917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:13:54.893006 10917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:35309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:43275
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:43275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:54.894275 10917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:54.895830 10917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:54.905405 10923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:54.905405 10922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:54.905407 10925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:54.905812 10917 server_base.cc:1034] running on GCE node
I20260807 15:13:54.906438 10917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:54.907714 10917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:54.908915 10917 hybrid_clock.cc:648] HybridClock initialized: now 1786115634908886 us; error 49 us; skew 500 ppm
I20260807 15:13:54.911306 10917 webserver.cc:460] Webserver started at http://127.15.226.126:42553/ using document root <none> and password file <none>
I20260807 15:13:54.911964 10917 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:54.912037 10917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:54.912279 10917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:13:54.914587 10917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "edb17a3e40c348099a33bbef4210c3a8"
format_stamp: "Formatted at 2026-08-07 15:13:54 on dist-test-slave-3kk6"
server_key: "88d85d0bbc462a11eecccdb79b2c3dce"
server_key_iv: "b2b44879edc1a2831f9e7915c91458cc"
server_key_version: "encryptionkey@0"
I20260807 15:13:54.915313 10917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "edb17a3e40c348099a33bbef4210c3a8"
format_stamp: "Formatted at 2026-08-07 15:13:54 on dist-test-slave-3kk6"
server_key: "88d85d0bbc462a11eecccdb79b2c3dce"
server_key_iv: "b2b44879edc1a2831f9e7915c91458cc"
server_key_version: "encryptionkey@0"
I20260807 15:13:54.919410 10917 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:13:54.922271 10934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:54.923869 10917 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:13:54.924052 10917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "edb17a3e40c348099a33bbef4210c3a8"
format_stamp: "Formatted at 2026-08-07 15:13:54 on dist-test-slave-3kk6"
server_key: "88d85d0bbc462a11eecccdb79b2c3dce"
server_key_iv: "b2b44879edc1a2831f9e7915c91458cc"
server_key_version: "encryptionkey@0"
I20260807 15:13:54.924197 10917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:54.950094 10917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:54.951025 10917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:54.951287 10917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:54.959707 10917 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:43275
I20260807 15:13:54.959712 11008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:43275 every 8 connection(s)
I20260807 15:13:54.960853 10917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:13:54.964794 11009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:54.968587 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 10917
I20260807 15:13:54.968786 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:13:54.969158 16265 external_mini_cluster.cc:1428] Setting key a2f27721966c003bc4e6e79db10617e4
I20260807 15:13:54.972491 11009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edb17a3e40c348099a33bbef4210c3a8: Bootstrap starting.
I20260807 15:13:54.975714 11009 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edb17a3e40c348099a33bbef4210c3a8: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:54.976745 11009 log.cc:824] T 00000000000000000000000000000000 P edb17a3e40c348099a33bbef4210c3a8: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:54.979655 11009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edb17a3e40c348099a33bbef4210c3a8: No bootstrap required, opened a new log
I20260807 15:13:54.980216 11014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:54.970923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43844 (local address 127.15.226.126:43275)
0807 15:13:54.971639 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:54.971656 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:54.971710 (+ 54us) server_negotiation.cc:407] Connection header received
0807 15:13:54.972406 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:54.972424 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:54.972949 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:54.973286 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:54.973701 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.974795 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.976183 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:54.976714 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:54.976853 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:54.978062 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:54.978068 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:54.978085 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:13:54.978532 (+ 447us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:54.978596 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:13:54.978617 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:54.978828 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:54.978977 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 15:13:54.979316 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":163,"threads_started":1}
I20260807 15:13:54.983669 11009 raft_consensus.cc:348] T 00000000000000000000000000000000 P edb17a3e40c348099a33bbef4210c3a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edb17a3e40c348099a33bbef4210c3a8" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 43275 } }
I20260807 15:13:54.983946 11009 raft_consensus.cc:374] T 00000000000000000000000000000000 P edb17a3e40c348099a33bbef4210c3a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:54.984011 11009 raft_consensus.cc:729] T 00000000000000000000000000000000 P edb17a3e40c348099a33bbef4210c3a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edb17a3e40c348099a33bbef4210c3a8, State: Initialized, Role: FOLLOWER
I20260807 15:13:54.984519 11009 consensus_queue.cc:260] T 00000000000000000000000000000000 P edb17a3e40c348099a33bbef4210c3a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edb17a3e40c348099a33bbef4210c3a8" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 43275 } }
I20260807 15:13:54.984683 11009 raft_consensus.cc:388] T 00000000000000000000000000000000 P edb17a3e40c348099a33bbef4210c3a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:13:54.984759 11009 raft_consensus.cc:482] T 00000000000000000000000000000000 P edb17a3e40c348099a33bbef4210c3a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:13:54.984895 11009 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edb17a3e40c348099a33bbef4210c3a8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:54.986042 11009 raft_consensus.cc:504] T 00000000000000000000000000000000 P edb17a3e40c348099a33bbef4210c3a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edb17a3e40c348099a33bbef4210c3a8" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 43275 } }
I20260807 15:13:54.986454 11009 leader_election.cc:302] T 00000000000000000000000000000000 P edb17a3e40c348099a33bbef4210c3a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edb17a3e40c348099a33bbef4210c3a8; no voters:
I20260807 15:13:54.986883 11009 leader_election.cc:288] T 00000000000000000000000000000000 P edb17a3e40c348099a33bbef4210c3a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:13:54.987092 11016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edb17a3e40c348099a33bbef4210c3a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:13:54.987394 11016 raft_consensus.cc:686] T 00000000000000000000000000000000 P edb17a3e40c348099a33bbef4210c3a8 [term 1 LEADER]: Becoming Leader. State: Replica: edb17a3e40c348099a33bbef4210c3a8, State: Running, Role: LEADER
I20260807 15:13:54.987766 11016 consensus_queue.cc:237] T 00000000000000000000000000000000 P edb17a3e40c348099a33bbef4210c3a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edb17a3e40c348099a33bbef4210c3a8" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 43275 } }
I20260807 15:13:54.988395 11009 sys_catalog.cc:564] T 00000000000000000000000000000000 P edb17a3e40c348099a33bbef4210c3a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:13:54.989763 11018 sys_catalog.cc:455] T 00000000000000000000000000000000 P edb17a3e40c348099a33bbef4210c3a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader edb17a3e40c348099a33bbef4210c3a8. Latest consensus state: current_term: 1 leader_uuid: "edb17a3e40c348099a33bbef4210c3a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edb17a3e40c348099a33bbef4210c3a8" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 43275 } } }
I20260807 15:13:54.989924 11018 sys_catalog.cc:458] T 00000000000000000000000000000000 P edb17a3e40c348099a33bbef4210c3a8 [sys.catalog]: This master's current role is: LEADER
I20260807 15:13:54.990278 11017 sys_catalog.cc:455] T 00000000000000000000000000000000 P edb17a3e40c348099a33bbef4210c3a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "edb17a3e40c348099a33bbef4210c3a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edb17a3e40c348099a33bbef4210c3a8" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 43275 } } }
I20260807 15:13:54.990428 11017 sys_catalog.cc:458] T 00000000000000000000000000000000 P edb17a3e40c348099a33bbef4210c3a8 [sys.catalog]: This master's current role is: LEADER
I20260807 15:13:54.990351 11026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:13:54.993691 11026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:13:54.999806 11026 catalog_manager.cc:1347] Generated new cluster ID: 74dd63ed5fce40cba337cfcc9b34c2e7
I20260807 15:13:54.999917 11026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:13:55.022573 11026 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:13:55.024039 11026 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:13:55.030623 11026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P edb17a3e40c348099a33bbef4210c3a8: Generated new TSK 0
I20260807 15:13:55.031545 11026 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:13:55.039112 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:43275
--builtin_ntp_servers=127.15.226.84:35309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:13:55.163990 11043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:55.164314 11043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:55.164356 11043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:55.168854 11043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:55.168941 11043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:55.168999 11043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:13:55.174212 11043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:35309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:43275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:55.175494 11043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:55.177119 11043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:55.187750 11050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:55.187788 11052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:55.187745 11055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:55.188225 11043 server_base.cc:1034] running on GCE node
I20260807 15:13:55.188758 11043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:55.189675 11043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:55.190899 11043 hybrid_clock.cc:648] HybridClock initialized: now 1786115635190885 us; error 33 us; skew 500 ppm
I20260807 15:13:55.193168 11043 webserver.cc:460] Webserver started at http://127.15.226.65:46715/ using document root <none> and password file <none>
I20260807 15:13:55.193807 11043 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:55.193884 11043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:55.194078 11043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:13:55.196535 11043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "ed2acbecff1c49ab8b6f041241a0926e"
format_stamp: "Formatted at 2026-08-07 15:13:55 on dist-test-slave-3kk6"
server_key: "eccb468f268b1e1ccf29ad26695769f0"
server_key_iv: "b4d4870641b922eff6ab7a4221392024"
server_key_version: "encryptionkey@0"
I20260807 15:13:55.197162 11043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "ed2acbecff1c49ab8b6f041241a0926e"
format_stamp: "Formatted at 2026-08-07 15:13:55 on dist-test-slave-3kk6"
server_key: "eccb468f268b1e1ccf29ad26695769f0"
server_key_iv: "b4d4870641b922eff6ab7a4221392024"
server_key_version: "encryptionkey@0"
I20260807 15:13:55.201556 11043 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260807 15:13:55.204691 11063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:55.206174 11043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:13:55.206301 11043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "ed2acbecff1c49ab8b6f041241a0926e"
format_stamp: "Formatted at 2026-08-07 15:13:55 on dist-test-slave-3kk6"
server_key: "eccb468f268b1e1ccf29ad26695769f0"
server_key_iv: "b4d4870641b922eff6ab7a4221392024"
server_key_version: "encryptionkey@0"
I20260807 15:13:55.206423 11043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:55.230711 11043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:55.231545 11043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:55.231761 11043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:55.232406 11043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:55.233484 11043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:55.233538 11043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:55.233583 11043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:55.233599 11043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:55.244508 11043 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:46607
I20260807 15:13:55.244527 11240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:46607 every 8 connection(s)
I20260807 15:13:55.245590 11043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:13:55.248442 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 11043
I20260807 15:13:55.248600 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:13:55.248994 16265 external_mini_cluster.cc:1428] Setting key c6e16ca50ca13436e503870c437d43da
I20260807 15:13:55.251144 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:43275
--builtin_ntp_servers=127.15.226.84:35309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 15:13:55.256822 11014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:55.247722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:36379 (local address 127.15.226.126:43275)
0807 15:13:55.247947 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:55.247951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:55.249348 (+ 1397us) server_negotiation.cc:407] Connection header received
0807 15:13:55.251059 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:55.251063 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:55.251124 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:55.251246 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:55.252367 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.252986 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.254408 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.254734 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.254868 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:55.256225 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:55.256229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:55.256232 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:13:55.256249 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:55.256280 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:13:55.256290 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:55.256382 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:55.256566 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 15:13:55.256643 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260807 15:13:55.258028 11243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:55.248143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:43275 (local address 127.15.226.65:36379)
0807 15:13:55.249157 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:55.249208 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:55.250812 (+ 1604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:55.251479 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:55.251488 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:55.251589 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:55.252183 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:55.252198 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.253175 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.253178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:55.254245 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:55.254257 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.254923 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.254928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:55.255130 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:55.255906 (+ 776us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:13:55.255908 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:55.255958 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:13:55.256037 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:55.256384 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:55.256399 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:55.256407 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 15:13:55.256566 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 15:13:55.256926 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":186,"threads_started":1}
I20260807 15:13:55.259734 11241 heartbeater.cc:347] Connected to a master server at 127.15.226.126:43275
I20260807 15:13:55.260025 11241 heartbeater.cc:464] Registering TS with master...
I20260807 15:13:55.260670 11241 heartbeater.cc:511] Master 127.15.226.126:43275 requested a full tablet report, sending...
I20260807 15:13:55.262741 10953 ts_manager.cc:194] Registered new tserver with Master: ed2acbecff1c49ab8b6f041241a0926e (127.15.226.65:46607)
I20260807 15:13:55.264618 10953 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.226.65:36379
W20260807 15:13:55.375926 11247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:55.376240 11247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:55.376281 11247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:55.380468 11247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:55.380546 11247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:55.380607 11247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:13:55.385670 11247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:35309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:43275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:55.386914 11247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:55.388473 11247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:55.399312 11254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:55.399329 11257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:55.399372 11247 server_base.cc:1034] running on GCE node
W20260807 15:13:55.399343 11255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:55.400143 11247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:55.400898 11247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:55.402101 11247 hybrid_clock.cc:648] HybridClock initialized: now 1786115635402087 us; error 31 us; skew 500 ppm
I20260807 15:13:55.404343 11247 webserver.cc:460] Webserver started at http://127.15.226.66:45317/ using document root <none> and password file <none>
I20260807 15:13:55.404989 11247 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:55.405093 11247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:55.405350 11247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:13:55.407712 11247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "36a16634a7f74a23bac5613cabf6f5d2"
format_stamp: "Formatted at 2026-08-07 15:13:55 on dist-test-slave-3kk6"
server_key: "ca955389bacd5700d8353b36243feb98"
server_key_iv: "ef2164288720291a7117b1260de27fc6"
server_key_version: "encryptionkey@0"
I20260807 15:13:55.408407 11247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "36a16634a7f74a23bac5613cabf6f5d2"
format_stamp: "Formatted at 2026-08-07 15:13:55 on dist-test-slave-3kk6"
server_key: "ca955389bacd5700d8353b36243feb98"
server_key_iv: "ef2164288720291a7117b1260de27fc6"
server_key_version: "encryptionkey@0"
I20260807 15:13:55.412616 11247 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:13:55.415361 11265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:55.416699 11247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:13:55.416900 11247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "36a16634a7f74a23bac5613cabf6f5d2"
format_stamp: "Formatted at 2026-08-07 15:13:55 on dist-test-slave-3kk6"
server_key: "ca955389bacd5700d8353b36243feb98"
server_key_iv: "ef2164288720291a7117b1260de27fc6"
server_key_version: "encryptionkey@0"
I20260807 15:13:55.417043 11247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:55.452836 11247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:55.453709 11247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:55.453971 11247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:55.454701 11247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:55.455781 11247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:55.455858 11247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:55.455938 11247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:55.455989 11247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:55.466135 11247 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:44323
I20260807 15:13:55.466154 11443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:44323 every 8 connection(s)
I20260807 15:13:55.467201 11247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:13:55.470247 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 11247
I20260807 15:13:55.470387 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:13:55.470746 16265 external_mini_cluster.cc:1428] Setting key e0bf79a390e77d2af21f111c0e15c1b2
I20260807 15:13:55.473301 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:43275
--builtin_ntp_servers=127.15.226.84:35309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 15:13:55.479430 11014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:55.469208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:45773 (local address 127.15.226.126:43275)
0807 15:13:55.469383 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:55.469386 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:55.470453 (+ 1067us) server_negotiation.cc:407] Connection header received
0807 15:13:55.472591 (+ 2138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:55.472595 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:55.472653 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:55.472822 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:55.473920 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.474834 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.476735 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.477199 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.477338 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:55.478692 (+ 1354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:55.478700 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:55.478706 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:13:55.478729 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:55.478766 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:13:55.478781 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:55.478871 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:55.479111 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 15:13:55.479198 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 15:13:55.480516 11450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:55.469508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:43275 (local address 127.15.226.66:45773)
0807 15:13:55.470242 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:55.470287 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:55.472192 (+ 1905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:55.472989 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:55.472996 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:55.473092 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:55.473755 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:55.473772 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.474992 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.474996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:55.476565 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:55.476581 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.477399 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.477402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:55.477570 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:55.478329 (+ 759us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:13:55.478330 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:55.478379 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:13:55.478463 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:55.478894 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:55.478906 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:55.478913 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 15:13:55.479077 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 15:13:55.479470 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":131,"threads_started":1}
I20260807 15:13:55.482015 11447 heartbeater.cc:347] Connected to a master server at 127.15.226.126:43275
I20260807 15:13:55.482299 11447 heartbeater.cc:464] Registering TS with master...
I20260807 15:13:55.482993 11447 heartbeater.cc:511] Master 127.15.226.126:43275 requested a full tablet report, sending...
I20260807 15:13:55.484238 10953 ts_manager.cc:194] Registered new tserver with Master: 36a16634a7f74a23bac5613cabf6f5d2 (127.15.226.66:44323)
I20260807 15:13:55.484907 10953 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.226.66:45773
W20260807 15:13:55.599150 11451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:55.599486 11451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:55.599596 11451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:55.603995 11451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:55.604099 11451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:55.604194 11451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:13:55.609325 11451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:35309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:43275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:55.610618 11451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:55.612251 11451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:55.622505 11459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:55.622512 11462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:55.622519 11458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:55.622738 11451 server_base.cc:1034] running on GCE node
I20260807 15:13:55.623409 11451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:55.624119 11451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:55.625300 11451 hybrid_clock.cc:648] HybridClock initialized: now 1786115635625281 us; error 34 us; skew 500 ppm
I20260807 15:13:55.627467 11451 webserver.cc:460] Webserver started at http://127.15.226.67:37547/ using document root <none> and password file <none>
I20260807 15:13:55.628145 11451 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:55.628216 11451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:55.628468 11451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:13:55.630795 11451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "667824377bc045ccbe9034458863d7b9"
format_stamp: "Formatted at 2026-08-07 15:13:55 on dist-test-slave-3kk6"
server_key: "08f10e28c3f8ec312028aa5b4c63aad1"
server_key_iv: "d0721e2558d14f513b8bc39dcb484132"
server_key_version: "encryptionkey@0"
I20260807 15:13:55.631413 11451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "667824377bc045ccbe9034458863d7b9"
format_stamp: "Formatted at 2026-08-07 15:13:55 on dist-test-slave-3kk6"
server_key: "08f10e28c3f8ec312028aa5b4c63aad1"
server_key_iv: "d0721e2558d14f513b8bc39dcb484132"
server_key_version: "encryptionkey@0"
I20260807 15:13:55.635380 11451 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:13:55.638154 11469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:55.639443 11451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:13:55.639583 11451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "667824377bc045ccbe9034458863d7b9"
format_stamp: "Formatted at 2026-08-07 15:13:55 on dist-test-slave-3kk6"
server_key: "08f10e28c3f8ec312028aa5b4c63aad1"
server_key_iv: "d0721e2558d14f513b8bc39dcb484132"
server_key_version: "encryptionkey@0"
I20260807 15:13:55.639745 11451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:55.663489 11451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:55.664266 11451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:55.664443 11451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:55.665071 11451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:55.666087 11451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:55.666139 11451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:55.666184 11451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:55.666200 11451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:55.676690 11451 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:44305
I20260807 15:13:55.676716 11634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:44305 every 8 connection(s)
I20260807 15:13:55.677762 11451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:13:55.683022 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 11451
I20260807 15:13:55.683148 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:13:55.683529 16265 external_mini_cluster.cc:1428] Setting key 22db2402e9d2c61b0a028071664980fb
I20260807 15:13:55.688864 11014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:55.680033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:34785 (local address 127.15.226.126:43275)
0807 15:13:55.680232 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:55.680236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:55.681232 (+ 996us) server_negotiation.cc:407] Connection header received
0807 15:13:55.682843 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:55.682846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:55.682905 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:55.682993 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:55.684283 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.685070 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.686500 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.686876 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.686978 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:55.688349 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:55.688353 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:55.688356 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:13:55.688379 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:55.688400 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:13:55.688408 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:55.688473 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:55.688631 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 15:13:55.688692 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 15:13:55.689955 11638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:55.680297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:43275 (local address 127.15.226.67:34785)
0807 15:13:55.681053 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:55.681101 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:55.682537 (+ 1436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:55.683203 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:55.683214 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:55.683332 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:55.684093 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:55.684115 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.685205 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.685208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:55.686343 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:55.686355 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.687043 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.687046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:55.687210 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:55.687968 (+ 758us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:13:55.687988 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:55.688055 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:13:55.688148 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:55.688491 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:55.688499 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:55.688503 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 15:13:55.688614 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 15:13:55.688916 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":144,"threads_started":1}
I20260807 15:13:55.691529 11635 heartbeater.cc:347] Connected to a master server at 127.15.226.126:43275
I20260807 15:13:55.691839 11635 heartbeater.cc:464] Registering TS with master...
I20260807 15:13:55.692499 11635 heartbeater.cc:511] Master 127.15.226.126:43275 requested a full tablet report, sending...
I20260807 15:13:55.693766 10953 ts_manager.cc:194] Registered new tserver with Master: 667824377bc045ccbe9034458863d7b9 (127.15.226.67:44305)
I20260807 15:13:55.694427 10953 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.226.67:34785
I20260807 15:13:55.697800 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:13:55.704282 11014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:55.699995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43856 (local address 127.15.226.126:43275)
0807 15:13:55.700164 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:55.700169 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:55.700370 (+ 201us) server_negotiation.cc:407] Connection header received
0807 15:13:55.700470 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:55.700473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:55.700518 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:55.700590 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:55.700936 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.701467 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.702343 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.702562 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.702700 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:55.703700 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:55.703702 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:55.703705 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:13:55.703717 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:55.703734 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:13:55.703741 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:55.703843 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:55.704030 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 15:13:55.704077 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 15:13:55.707686 10953 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43856:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:13:55.711004 10953 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:13:55.727229 11652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:55.721653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:44305 (local address 127.0.0.1:38772)
0807 15:13:55.722462 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:55.722476 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:55.722555 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:55.723096 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:55.723101 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:55.723120 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:55.723195 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:55.723201 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.724561 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.724564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:55.725550 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:55.725560 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.726764 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.726766 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:55.726858 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:55.726894 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 15:13:55.726989 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 15:13:55.727093 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"mutex_wait_us":38,"thread_start_us":108,"threads_started":1}
I20260807 15:13:55.727335 11656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:55.722115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38772 (local address 127.15.226.67:44305)
0807 15:13:55.722571 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:55.722583 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:55.722605 (+ 22us) server_negotiation.cc:407] Connection header received
0807 15:13:55.722779 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:55.722786 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:55.722967 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:55.723119 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:55.723322 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.724427 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.725693 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.726588 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.726763 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:55.726994 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:55.727049 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 15:13:55.727148 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":185,"threads_started":1}
I20260807 15:13:55.728389 11650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:55.720461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:46607 (local address 127.0.0.1:49134)
0807 15:13:55.720915 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:55.720983 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:55.721136 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:55.722028 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:55.722036 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:55.722064 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:55.722195 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:55.722200 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.723984 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.723989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:55.725706 (+ 1717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:55.725719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.727965 (+ 2246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.727969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:55.728064 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:55.728088 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 15:13:55.728162 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 15:13:55.728244 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":165,"thread_start_us":97,"threads_started":1}
I20260807 15:13:55.728503 11651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:55.720635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49134 (local address 127.15.226.65:46607)
0807 15:13:55.721151 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:55.721159 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:55.721189 (+ 30us) server_negotiation.cc:407] Connection header received
0807 15:13:55.721407 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:55.721413 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:55.721729 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:55.721899 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:55.722583 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.723818 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.725856 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.726589 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.727950 (+ 1361us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:55.728143 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:55.728211 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 15:13:55.728317 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":135,"threads_started":1}
I20260807 15:13:55.729869 11654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:55.721286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:44323 (local address 127.0.0.1:52374)
0807 15:13:55.722163 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:55.722180 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:55.722299 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:55.722823 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:55.722826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:55.722845 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:55.722937 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:55.722944 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.724052 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.724055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:55.725706 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:55.725719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.729420 (+ 3701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.729424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:55.729524 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:55.729552 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 15:13:55.729642 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 15:13:55.729730 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"mutex_wait_us":89,"thread_start_us":122,"threads_started":1}
I20260807 15:13:55.730116 11653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:55.721475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52374 (local address 127.15.226.66:44323)
0807 15:13:55.722002 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:55.722017 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:55.722302 (+ 285us) server_negotiation.cc:407] Connection header received
0807 15:13:55.722424 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:55.722429 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:55.722636 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:55.722853 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:55.723049 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.723912 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.726754 (+ 2842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.727563 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.727674 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:55.727893 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:55.729727 (+ 1834us) server_negotiation.cc:299] Negotiation successful
0807 15:13:55.729853 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":170,"threads_started":1}
I20260807 15:13:55.730942 11535 tablet_service.cc:1467] Processing CreateTablet for tablet e278afffe2734cafb059b7fe2733ccb1 (DEFAULT_TABLE table=test-table [id=ea439322b190413baa23ca8d1e3c7f5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:13:55.731300 11134 tablet_service.cc:1467] Processing CreateTablet for tablet e278afffe2734cafb059b7fe2733ccb1 (DEFAULT_TABLE table=test-table [id=ea439322b190413baa23ca8d1e3c7f5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:13:55.732251 11535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e278afffe2734cafb059b7fe2733ccb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:55.732398 11134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e278afffe2734cafb059b7fe2733ccb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:55.733124 11341 tablet_service.cc:1467] Processing CreateTablet for tablet e278afffe2734cafb059b7fe2733ccb1 (DEFAULT_TABLE table=test-table [id=ea439322b190413baa23ca8d1e3c7f5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:13:55.734261 11341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e278afffe2734cafb059b7fe2733ccb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:55.740018 11657 tablet_bootstrap.cc:492] T e278afffe2734cafb059b7fe2733ccb1 P ed2acbecff1c49ab8b6f041241a0926e: Bootstrap starting.
I20260807 15:13:55.740625 11658 tablet_bootstrap.cc:492] T e278afffe2734cafb059b7fe2733ccb1 P 667824377bc045ccbe9034458863d7b9: Bootstrap starting.
I20260807 15:13:55.742117 11660 tablet_bootstrap.cc:492] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2: Bootstrap starting.
I20260807 15:13:55.742441 11657 tablet_bootstrap.cc:654] T e278afffe2734cafb059b7fe2733ccb1 P ed2acbecff1c49ab8b6f041241a0926e: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:55.742919 11658 tablet_bootstrap.cc:654] T e278afffe2734cafb059b7fe2733ccb1 P 667824377bc045ccbe9034458863d7b9: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:55.743472 11657 log.cc:824] T e278afffe2734cafb059b7fe2733ccb1 P ed2acbecff1c49ab8b6f041241a0926e: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:55.743810 11658 log.cc:824] T e278afffe2734cafb059b7fe2733ccb1 P 667824377bc045ccbe9034458863d7b9: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:55.744575 11660 tablet_bootstrap.cc:654] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:55.745461 11660 log.cc:824] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:55.746001 11657 tablet_bootstrap.cc:492] T e278afffe2734cafb059b7fe2733ccb1 P ed2acbecff1c49ab8b6f041241a0926e: No bootstrap required, opened a new log
I20260807 15:13:55.746243 11657 ts_tablet_manager.cc:1397] T e278afffe2734cafb059b7fe2733ccb1 P ed2acbecff1c49ab8b6f041241a0926e: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 15:13:55.746361 11658 tablet_bootstrap.cc:492] T e278afffe2734cafb059b7fe2733ccb1 P 667824377bc045ccbe9034458863d7b9: No bootstrap required, opened a new log
I20260807 15:13:55.746559 11658 ts_tablet_manager.cc:1397] T e278afffe2734cafb059b7fe2733ccb1 P 667824377bc045ccbe9034458863d7b9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 15:13:55.747668 11660 tablet_bootstrap.cc:492] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2: No bootstrap required, opened a new log
I20260807 15:13:55.747864 11660 ts_tablet_manager.cc:1397] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 15:13:55.750145 11658 raft_consensus.cc:348] T e278afffe2734cafb059b7fe2733ccb1 P 667824377bc045ccbe9034458863d7b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed2acbecff1c49ab8b6f041241a0926e" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 46607 } } peers { permanent_uuid: "36a16634a7f74a23bac5613cabf6f5d2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44323 } } peers { permanent_uuid: "667824377bc045ccbe9034458863d7b9" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 44305 } }
I20260807 15:13:55.750491 11658 raft_consensus.cc:374] T e278afffe2734cafb059b7fe2733ccb1 P 667824377bc045ccbe9034458863d7b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:55.750563 11658 raft_consensus.cc:729] T e278afffe2734cafb059b7fe2733ccb1 P 667824377bc045ccbe9034458863d7b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 667824377bc045ccbe9034458863d7b9, State: Initialized, Role: FOLLOWER
I20260807 15:13:55.751019 11657 raft_consensus.cc:348] T e278afffe2734cafb059b7fe2733ccb1 P ed2acbecff1c49ab8b6f041241a0926e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed2acbecff1c49ab8b6f041241a0926e" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 46607 } } peers { permanent_uuid: "36a16634a7f74a23bac5613cabf6f5d2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44323 } } peers { permanent_uuid: "667824377bc045ccbe9034458863d7b9" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 44305 } }
I20260807 15:13:55.751164 11658 consensus_queue.cc:260] T e278afffe2734cafb059b7fe2733ccb1 P 667824377bc045ccbe9034458863d7b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed2acbecff1c49ab8b6f041241a0926e" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 46607 } } peers { permanent_uuid: "36a16634a7f74a23bac5613cabf6f5d2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44323 } } peers { permanent_uuid: "667824377bc045ccbe9034458863d7b9" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 44305 } }
I20260807 15:13:55.751286 11657 raft_consensus.cc:374] T e278afffe2734cafb059b7fe2733ccb1 P ed2acbecff1c49ab8b6f041241a0926e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:55.751345 11657 raft_consensus.cc:729] T e278afffe2734cafb059b7fe2733ccb1 P ed2acbecff1c49ab8b6f041241a0926e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed2acbecff1c49ab8b6f041241a0926e, State: Initialized, Role: FOLLOWER
I20260807 15:13:55.751924 11635 heartbeater.cc:503] Master 127.15.226.126:43275 was elected leader, sending a full tablet report...
I20260807 15:13:55.751850 11657 consensus_queue.cc:260] T e278afffe2734cafb059b7fe2733ccb1 P ed2acbecff1c49ab8b6f041241a0926e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed2acbecff1c49ab8b6f041241a0926e" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 46607 } } peers { permanent_uuid: "36a16634a7f74a23bac5613cabf6f5d2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44323 } } peers { permanent_uuid: "667824377bc045ccbe9034458863d7b9" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 44305 } }
I20260807 15:13:55.752311 11658 ts_tablet_manager.cc:1428] T e278afffe2734cafb059b7fe2733ccb1 P 667824377bc045ccbe9034458863d7b9: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 15:13:55.752801 11657 ts_tablet_manager.cc:1428] T e278afffe2734cafb059b7fe2733ccb1 P ed2acbecff1c49ab8b6f041241a0926e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 15:13:55.753384 11241 heartbeater.cc:503] Master 127.15.226.126:43275 was elected leader, sending a full tablet report...
I20260807 15:13:55.753405 11660 raft_consensus.cc:348] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed2acbecff1c49ab8b6f041241a0926e" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 46607 } } peers { permanent_uuid: "36a16634a7f74a23bac5613cabf6f5d2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44323 } } peers { permanent_uuid: "667824377bc045ccbe9034458863d7b9" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 44305 } }
I20260807 15:13:55.753680 11660 raft_consensus.cc:374] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:55.753756 11660 raft_consensus.cc:729] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36a16634a7f74a23bac5613cabf6f5d2, State: Initialized, Role: FOLLOWER
I20260807 15:13:55.754190 11660 consensus_queue.cc:260] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed2acbecff1c49ab8b6f041241a0926e" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 46607 } } peers { permanent_uuid: "36a16634a7f74a23bac5613cabf6f5d2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44323 } } peers { permanent_uuid: "667824377bc045ccbe9034458863d7b9" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 44305 } }
I20260807 15:13:55.755278 11660 ts_tablet_manager.cc:1428] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 15:13:55.756249 11447 heartbeater.cc:503] Master 127.15.226.126:43275 was elected leader, sending a full tablet report...
I20260807 15:13:55.815634 11672 raft_consensus.cc:482] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:13:55.815914 11672 raft_consensus.cc:504] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed2acbecff1c49ab8b6f041241a0926e" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 46607 } } peers { permanent_uuid: "36a16634a7f74a23bac5613cabf6f5d2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44323 } } peers { permanent_uuid: "667824377bc045ccbe9034458863d7b9" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 44305 } }
I20260807 15:13:55.817309 11672 leader_election.cc:288] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed2acbecff1c49ab8b6f041241a0926e (127.15.226.65:46607), 667824377bc045ccbe9034458863d7b9 (127.15.226.67:44305)
I20260807 15:13:55.822412 11450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:55.817618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:46607 (local address 127.15.226.66:34627)
0807 15:13:55.817770 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:55.817791 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:55.817910 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:55.818325 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:55.818329 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:55.818357 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:55.818523 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:55.818533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.819460 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.819464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:55.820902 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:55.820916 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.821897 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.821901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:55.822041 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:55.822070 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 15:13:55.822161 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 15:13:55.822249 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":75}
I20260807 15:13:55.822461 11651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:55.817799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:34627 (local address 127.15.226.65:46607)
0807 15:13:55.817959 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:55.817965 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:55.818024 (+ 59us) server_negotiation.cc:407] Connection header received
0807 15:13:55.818098 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:55.818101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:55.818164 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:55.818252 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:55.818699 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.819331 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.821054 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.821750 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.821885 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:55.821962 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:55.822199 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 15:13:55.822291 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 15:13:55.822628 11673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:55.817947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:44305 (local address 127.15.226.66:58069)
0807 15:13:55.818363 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:55.818377 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:55.818462 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:55.818713 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:55.818716 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:55.818736 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:55.818871 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:55.818877 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.819590 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.819593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:55.820902 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:55.820916 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.822092 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.822096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:55.822236 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:55.822264 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 15:13:55.822391 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 15:13:55.822465 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":31,"thread_start_us":176,"threads_started":1}
I20260807 15:13:55.822718 11656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:55.818142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:58069 (local address 127.15.226.67:44305)
0807 15:13:55.818463 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:55.818468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:55.818485 (+ 17us) server_negotiation.cc:407] Connection header received
0807 15:13:55.818541 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:55.818544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:55.818586 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:55.818710 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:55.818996 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.819487 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.821189 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.821853 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.822004 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:55.822060 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:55.822437 (+ 377us) server_negotiation.cc:299] Negotiation successful
0807 15:13:55.822539 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227}
I20260807 15:13:55.823531 11569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e278afffe2734cafb059b7fe2733ccb1" candidate_uuid: "36a16634a7f74a23bac5613cabf6f5d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "667824377bc045ccbe9034458863d7b9" is_pre_election: true
I20260807 15:13:55.823530 11171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e278afffe2734cafb059b7fe2733ccb1" candidate_uuid: "36a16634a7f74a23bac5613cabf6f5d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed2acbecff1c49ab8b6f041241a0926e" is_pre_election: true
I20260807 15:13:55.823872 11171 raft_consensus.cc:2454] T e278afffe2734cafb059b7fe2733ccb1 P ed2acbecff1c49ab8b6f041241a0926e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36a16634a7f74a23bac5613cabf6f5d2 in term 0.
I20260807 15:13:55.823872 11569 raft_consensus.cc:2454] T e278afffe2734cafb059b7fe2733ccb1 P 667824377bc045ccbe9034458863d7b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36a16634a7f74a23bac5613cabf6f5d2 in term 0.
I20260807 15:13:55.824455 11268 leader_election.cc:302] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36a16634a7f74a23bac5613cabf6f5d2, 667824377bc045ccbe9034458863d7b9; no voters:
I20260807 15:13:55.824836 11672 raft_consensus.cc:2781] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:13:55.824937 11672 raft_consensus.cc:482] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:13:55.824970 11672 raft_consensus.cc:3037] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:55.826246 11672 raft_consensus.cc:504] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed2acbecff1c49ab8b6f041241a0926e" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 46607 } } peers { permanent_uuid: "36a16634a7f74a23bac5613cabf6f5d2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44323 } } peers { permanent_uuid: "667824377bc045ccbe9034458863d7b9" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 44305 } }
I20260807 15:13:55.826668 11672 leader_election.cc:288] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2 [CANDIDATE]: Term 1 election: Requested vote from peers ed2acbecff1c49ab8b6f041241a0926e (127.15.226.65:46607), 667824377bc045ccbe9034458863d7b9 (127.15.226.67:44305)
I20260807 15:13:55.827198 11171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e278afffe2734cafb059b7fe2733ccb1" candidate_uuid: "36a16634a7f74a23bac5613cabf6f5d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed2acbecff1c49ab8b6f041241a0926e"
I20260807 15:13:55.827369 11569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e278afffe2734cafb059b7fe2733ccb1" candidate_uuid: "36a16634a7f74a23bac5613cabf6f5d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "667824377bc045ccbe9034458863d7b9"
I20260807 15:13:55.827389 11171 raft_consensus.cc:3037] T e278afffe2734cafb059b7fe2733ccb1 P ed2acbecff1c49ab8b6f041241a0926e [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:55.827533 11569 raft_consensus.cc:3037] T e278afffe2734cafb059b7fe2733ccb1 P 667824377bc045ccbe9034458863d7b9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:55.828758 11171 raft_consensus.cc:2454] T e278afffe2734cafb059b7fe2733ccb1 P ed2acbecff1c49ab8b6f041241a0926e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36a16634a7f74a23bac5613cabf6f5d2 in term 1.
I20260807 15:13:55.828890 11569 raft_consensus.cc:2454] T e278afffe2734cafb059b7fe2733ccb1 P 667824377bc045ccbe9034458863d7b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36a16634a7f74a23bac5613cabf6f5d2 in term 1.
I20260807 15:13:55.829272 11268 leader_election.cc:302] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36a16634a7f74a23bac5613cabf6f5d2, 667824377bc045ccbe9034458863d7b9; no voters:
I20260807 15:13:55.829479 11672 raft_consensus.cc:2781] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:13:55.829793 11672 raft_consensus.cc:686] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2 [term 1 LEADER]: Becoming Leader. State: Replica: 36a16634a7f74a23bac5613cabf6f5d2, State: Running, Role: LEADER
I20260807 15:13:55.830191 11672 consensus_queue.cc:237] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed2acbecff1c49ab8b6f041241a0926e" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 46607 } } peers { permanent_uuid: "36a16634a7f74a23bac5613cabf6f5d2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44323 } } peers { permanent_uuid: "667824377bc045ccbe9034458863d7b9" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 44305 } }
I20260807 15:13:55.833922 10953 catalog_manager.cc:5495] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36a16634a7f74a23bac5613cabf6f5d2 (127.15.226.66). New cstate: current_term: 1 leader_uuid: "36a16634a7f74a23bac5613cabf6f5d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed2acbecff1c49ab8b6f041241a0926e" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 46607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36a16634a7f74a23bac5613cabf6f5d2" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 44323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "667824377bc045ccbe9034458863d7b9" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 44305 } health_report { overall_health: UNKNOWN } } }
I20260807 15:13:55.855340 11653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:55.851637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52390 (local address 127.15.226.66:44323)
0807 15:13:55.851801 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:55.851805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:55.851819 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:13:55.851895 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:55.851898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:55.851956 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:55.852052 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:13:55.852290 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.852810 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.853678 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:55.853888 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:55.853969 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:55.854246 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:13:55.854776 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:13:55.854903 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:55.855139 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 15:13:55.855180 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 15:13:55.862179 11569 raft_consensus.cc:1261] T e278afffe2734cafb059b7fe2733ccb1 P 667824377bc045ccbe9034458863d7b9 [term 1 FOLLOWER]: Refusing update from remote peer 36a16634a7f74a23bac5613cabf6f5d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:13:55.862308 11171 raft_consensus.cc:1261] T e278afffe2734cafb059b7fe2733ccb1 P ed2acbecff1c49ab8b6f041241a0926e [term 1 FOLLOWER]: Refusing update from remote peer 36a16634a7f74a23bac5613cabf6f5d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:13:55.863147 11672 consensus_queue.cc:1035] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "667824377bc045ccbe9034458863d7b9" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 44305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:13:55.863422 11676 consensus_queue.cc:1035] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed2acbecff1c49ab8b6f041241a0926e" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 46607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:13:55.882285 10951 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:13:55.882491 10951 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:13:55.885293 10951 catalog_manager.cc:5782] T 00000000000000000000000000000000 P edb17a3e40c348099a33bbef4210c3a8: Sending DeleteTablet for 3 replicas of tablet e278afffe2734cafb059b7fe2733ccb1
I20260807 15:13:55.886423 11535 tablet_service.cc:1514] Processing DeleteTablet for tablet e278afffe2734cafb059b7fe2733ccb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:13:55 UTC) from {username='slave'} at 127.0.0.1:38772
I20260807 15:13:55.886428 11134 tablet_service.cc:1514] Processing DeleteTablet for tablet e278afffe2734cafb059b7fe2733ccb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:13:55 UTC) from {username='slave'} at 127.0.0.1:49134
I20260807 15:13:55.886605 11341 tablet_service.cc:1514] Processing DeleteTablet for tablet e278afffe2734cafb059b7fe2733ccb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:13:55 UTC) from {username='slave'} at 127.0.0.1:52374
I20260807 15:13:55.887089 11694 tablet_replica.cc:330] T e278afffe2734cafb059b7fe2733ccb1 P 667824377bc045ccbe9034458863d7b9: stopping tablet replica
I20260807 15:13:55.887149 11695 tablet_replica.cc:330] T e278afffe2734cafb059b7fe2733ccb1 P ed2acbecff1c49ab8b6f041241a0926e: stopping tablet replica
I20260807 15:13:55.887234 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 11043
I20260807 15:13:55.887485 11695 raft_consensus.cc:2229] T e278afffe2734cafb059b7fe2733ccb1 P ed2acbecff1c49ab8b6f041241a0926e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:13:55.887486 11694 raft_consensus.cc:2229] T e278afffe2734cafb059b7fe2733ccb1 P 667824377bc045ccbe9034458863d7b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:13:55.887882 11694 raft_consensus.cc:2258] T e278afffe2734cafb059b7fe2733ccb1 P 667824377bc045ccbe9034458863d7b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:13:55.888613 11697 tablet_replica.cc:330] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2: stopping tablet replica
I20260807 15:13:55.888964 11697 raft_consensus.cc:2229] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:13:55.889467 11697 raft_consensus.cc:2258] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:13:55.892453 11694 ts_tablet_manager.cc:1905] T e278afffe2734cafb059b7fe2733ccb1 P 667824377bc045ccbe9034458863d7b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:13:55.895494 11697 ts_tablet_manager.cc:1905] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:13:55.896055 11694 ts_tablet_manager.cc:1918] T e278afffe2734cafb059b7fe2733ccb1 P 667824377bc045ccbe9034458863d7b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:13:55.896193 11694 log.cc:1196] T e278afffe2734cafb059b7fe2733ccb1 P 667824377bc045ccbe9034458863d7b9: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/e278afffe2734cafb059b7fe2733ccb1
I20260807 15:13:55.896559 11694 ts_tablet_manager.cc:1939] T e278afffe2734cafb059b7fe2733ccb1 P 667824377bc045ccbe9034458863d7b9: Deleting consensus metadata
I20260807 15:13:55.898248 10937 catalog_manager.cc:4841] TS 667824377bc045ccbe9034458863d7b9 (127.15.226.67:44305): tablet e278afffe2734cafb059b7fe2733ccb1 (table test-table [id=ea439322b190413baa23ca8d1e3c7f5e]) successfully deleted
I20260807 15:13:55.898718 11697 ts_tablet_manager.cc:1918] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:13:55.898849 11697 log.cc:1196] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/e278afffe2734cafb059b7fe2733ccb1
I20260807 15:13:55.899237 11697 ts_tablet_manager.cc:1939] T e278afffe2734cafb059b7fe2733ccb1 P 36a16634a7f74a23bac5613cabf6f5d2: Deleting consensus metadata
I20260807 15:13:55.901955 10937 catalog_manager.cc:4841] TS 36a16634a7f74a23bac5613cabf6f5d2 (127.15.226.66:44323): tablet e278afffe2734cafb059b7fe2733ccb1 (table test-table [id=ea439322b190413baa23ca8d1e3c7f5e]) successfully deleted
W20260807 15:13:55.909789 10937 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.226.65:46607 (error 108)
I20260807 15:13:55.911000 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 11247
I20260807 15:13:55.911248 11652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:55.910872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:46607 (local address 127.0.0.1:49138)
0807 15:13:55.911025 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:55.911131 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.65:46607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 15:13:55.915148 10937 catalog_manager.cc:4584] TS ed2acbecff1c49ab8b6f041241a0926e (127.15.226.65:46607): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e278afffe2734cafb059b7fe2733ccb1: Network error: Client connection negotiation failed: client connection to 127.15.226.65:46607: connect: Connection refused (error 111)
I20260807 15:13:55.931911 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 11451
I20260807 15:13:55.949230 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 10917
2026-08-07T15:13:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1228 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:13:56 dist-test-slave-3kk6 krb5kdc[11701](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:13:56 dist-test-slave-3kk6 krb5kdc[11701](info): set up 2 sockets
Aug 07 15:13:56 dist-test-slave-3kk6 krb5kdc[11701](info): commencing operation
krb5kdc: starting...
W20260807 15:13:57.204962 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:13:57 dist-test-slave-3kk6 krb5kdc[11701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:13:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:13:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:13:57.346875 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:33279
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:34173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:33279
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:13:57.465282 11720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:57.465607 11720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:57.465649 11720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:57.470582 11720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:13:57.470721 11720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:57.470775 11720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:57.470808 11720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:13:57.470842 11720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:13:57.476631 11720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:34173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:33279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:33279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:57.477883 11720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:57.479094 11720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:57.488257 11731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:57.488314 11720 server_base.cc:1034] running on GCE node
W20260807 15:13:57.488261 11726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:57.488257 11729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:57.489176 11720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:57.490490 11720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:57.491720 11720 hybrid_clock.cc:648] HybridClock initialized: now 1786115637491700 us; error 41 us; skew 500 ppm
Aug 07 15:13:57 dist-test-slave-3kk6 krb5kdc[11701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115637, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:57.494966 11720 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:13:57.495982 11720 webserver.cc:460] Webserver started at http://127.15.226.126:41977/ using document root <none> and password file <none>
I20260807 15:13:57.496611 11720 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:57.496716 11720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:57.496973 11720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:13:57.499394 11720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "6326ba4bf28248399f9f193a781a1391"
format_stamp: "Formatted at 2026-08-07 15:13:57 on dist-test-slave-3kk6"
server_key: "6b864257a370430de8d7488bf4975916"
server_key_iv: "b68787722b22bed8026447787e633fc2"
server_key_version: "encryptionkey@0"
I20260807 15:13:57.500101 11720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "6326ba4bf28248399f9f193a781a1391"
format_stamp: "Formatted at 2026-08-07 15:13:57 on dist-test-slave-3kk6"
server_key: "6b864257a370430de8d7488bf4975916"
server_key_iv: "b68787722b22bed8026447787e633fc2"
server_key_version: "encryptionkey@0"
I20260807 15:13:57.504426 11720 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:13:57.507455 11746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:57.508846 11720 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:13:57.509033 11720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "6326ba4bf28248399f9f193a781a1391"
format_stamp: "Formatted at 2026-08-07 15:13:57 on dist-test-slave-3kk6"
server_key: "6b864257a370430de8d7488bf4975916"
server_key_iv: "b68787722b22bed8026447787e633fc2"
server_key_version: "encryptionkey@0"
I20260807 15:13:57.509176 11720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:57.522284 11720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:57.526094 11720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:57.526341 11720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:57.534698 11720 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:33279
I20260807 15:13:57.534750 11822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:33279 every 8 connection(s)
I20260807 15:13:57.535840 11720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:13:57.538926 11823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:57.545053 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 11720
I20260807 15:13:57.545138 11823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6326ba4bf28248399f9f193a781a1391: Bootstrap starting.
I20260807 15:13:57.545257 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:13:57.545554 16265 external_mini_cluster.cc:1428] Setting key 41ac687d895a6927c2fd62a1debd733c
I20260807 15:13:57.548054 11823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6326ba4bf28248399f9f193a781a1391: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:57.548857 11823 log.cc:824] T 00000000000000000000000000000000 P 6326ba4bf28248399f9f193a781a1391: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:57.551210 11823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6326ba4bf28248399f9f193a781a1391: No bootstrap required, opened a new log
Aug 07 15:13:57 dist-test-slave-3kk6 krb5kdc[11701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:13:57.555066 11823 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6326ba4bf28248399f9f193a781a1391 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6326ba4bf28248399f9f193a781a1391" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 33279 } }
I20260807 15:13:57.555413 11823 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6326ba4bf28248399f9f193a781a1391 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:57.555491 11823 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6326ba4bf28248399f9f193a781a1391 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6326ba4bf28248399f9f193a781a1391, State: Initialized, Role: FOLLOWER
I20260807 15:13:57.556022 11823 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6326ba4bf28248399f9f193a781a1391 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6326ba4bf28248399f9f193a781a1391" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 33279 } }
I20260807 15:13:57.556169 11823 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6326ba4bf28248399f9f193a781a1391 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:13:57.556259 11823 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6326ba4bf28248399f9f193a781a1391 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:13:57.556398 11823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6326ba4bf28248399f9f193a781a1391 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:57.557629 11823 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6326ba4bf28248399f9f193a781a1391 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6326ba4bf28248399f9f193a781a1391" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 33279 } }
I20260807 15:13:57.558032 11823 leader_election.cc:302] T 00000000000000000000000000000000 P 6326ba4bf28248399f9f193a781a1391 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6326ba4bf28248399f9f193a781a1391; no voters:
I20260807 15:13:57.558467 11823 leader_election.cc:288] T 00000000000000000000000000000000 P 6326ba4bf28248399f9f193a781a1391 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:13:57.558602 11835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6326ba4bf28248399f9f193a781a1391 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:13:57.558887 11835 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6326ba4bf28248399f9f193a781a1391 [term 1 LEADER]: Becoming Leader. State: Replica: 6326ba4bf28248399f9f193a781a1391, State: Running, Role: LEADER
I20260807 15:13:57.559293 11835 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6326ba4bf28248399f9f193a781a1391 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6326ba4bf28248399f9f193a781a1391" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 33279 } }
I20260807 15:13:57.559943 11823 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6326ba4bf28248399f9f193a781a1391 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:13:57.560578 11838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6326ba4bf28248399f9f193a781a1391 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6326ba4bf28248399f9f193a781a1391. Latest consensus state: current_term: 1 leader_uuid: "6326ba4bf28248399f9f193a781a1391" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6326ba4bf28248399f9f193a781a1391" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 33279 } } }
I20260807 15:13:57.560717 11838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6326ba4bf28248399f9f193a781a1391 [sys.catalog]: This master's current role is: LEADER
I20260807 15:13:57.561467 11837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6326ba4bf28248399f9f193a781a1391 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6326ba4bf28248399f9f193a781a1391" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6326ba4bf28248399f9f193a781a1391" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 33279 } } }
I20260807 15:13:57.561573 11837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6326ba4bf28248399f9f193a781a1391 [sys.catalog]: This master's current role is: LEADER
I20260807 15:13:57.561811 11829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:57.547097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34256 (local address 127.15.226.126:33279)
0807 15:13:57.547582 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:57.547599 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:57.547639 (+ 40us) server_negotiation.cc:407] Connection header received
0807 15:13:57.548517 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:57.548537 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:57.548883 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:57.549231 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:57.550114 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:57.551222 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:57.552197 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:57.552704 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:57.552896 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:57.555936 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:57.555942 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:57.555957 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:57.555988 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:57.558007 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:57.558533 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:57.558539 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:57.558546 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:57.558750 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:57.559109 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:57.559118 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:57.559119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:57.559637 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:57.559867 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:57.560391 (+ 524us) server_negotiation.cc:299] Negotiation successful
0807 15:13:57.560774 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":160,"threads_started":1}
I20260807 15:13:57.561882 11843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:13:57.565344 11843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:13:57.572249 11843 catalog_manager.cc:1347] Generated new cluster ID: 4de7bba6d24f4f7592dd83f2ddf1a3bd
I20260807 15:13:57.572352 11843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:13:57.585897 11843 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:13:57.587327 11843 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:13:57.597785 11843 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6326ba4bf28248399f9f193a781a1391: Generated new TSK 0
I20260807 15:13:57.598771 11843 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.226.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
Entry for principal kudu/127.15.226.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab.
I20260807 15:13:57.643038 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:33279
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:34173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:13:57.765928 11866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:57.766237 11866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:57.766283 11866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:57.770818 11866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:57.770898 11866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:57.770964 11866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:13:57.776569 11866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:34173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.65.keytab
--principal=kudu/127.15.226.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:33279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:57.777755 11866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:57.778981 11866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:13:57.789390 11866 server_base.cc:1034] running on GCE node
W20260807 15:13:57.789362 11873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:57.789343 11877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:57.789336 11875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:57.790063 11866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:57.790853 11866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:57.792048 11866 hybrid_clock.cc:648] HybridClock initialized: now 1786115637792031 us; error 33 us; skew 500 ppm
Aug 07 15:13:57 dist-test-slave-3kk6 krb5kdc[11701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115637, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:57.795051 11866 init.cc:377] Logged in from keytab as kudu/127.15.226.65@KRBTEST.COM (short username kudu)
I20260807 15:13:57.796013 11866 webserver.cc:460] Webserver started at http://127.15.226.65:37377/ using document root <none> and password file <none>
I20260807 15:13:57.796612 11866 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:57.796707 11866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:57.796960 11866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:13:57.799249 11866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "b7b4cbfcbff945f6a6df450584bd9a0b"
format_stamp: "Formatted at 2026-08-07 15:13:57 on dist-test-slave-3kk6"
server_key: "bf716ad3045b6bad5a3965258bc74917"
server_key_iv: "63034b986685c06fa8198fe747d100c3"
server_key_version: "encryptionkey@0"
I20260807 15:13:57.799889 11866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "b7b4cbfcbff945f6a6df450584bd9a0b"
format_stamp: "Formatted at 2026-08-07 15:13:57 on dist-test-slave-3kk6"
server_key: "bf716ad3045b6bad5a3965258bc74917"
server_key_iv: "63034b986685c06fa8198fe747d100c3"
server_key_version: "encryptionkey@0"
I20260807 15:13:57.804021 11866 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:13:57.806641 11887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:57.808120 11866 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:13:57.808277 11866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "b7b4cbfcbff945f6a6df450584bd9a0b"
format_stamp: "Formatted at 2026-08-07 15:13:57 on dist-test-slave-3kk6"
server_key: "bf716ad3045b6bad5a3965258bc74917"
server_key_iv: "63034b986685c06fa8198fe747d100c3"
server_key_version: "encryptionkey@0"
I20260807 15:13:57.808414 11866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:57.823624 11866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:57.827589 11866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:57.827850 11866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:57.828531 11866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:57.829479 11866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:57.829553 11866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:57.829628 11866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:57.829671 11866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:57.839299 11866 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:42167
I20260807 15:13:57.839327 12051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:42167 every 8 connection(s)
I20260807 15:13:57.840271 11866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:13:57.841428 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 11866
I20260807 15:13:57.841588 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:13:57.841992 16265 external_mini_cluster.cc:1428] Setting key 955b40f92e71418770134f0fa1ed633d
Aug 07 15:13:57 dist-test-slave-3kk6 krb5kdc[11701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115637, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.65@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.226.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.66@KRBTEST.COM" created.
I20260807 15:13:57.857473 11829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:57.842457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:52367 (local address 127.15.226.126:33279)
0807 15:13:57.842645 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:57.842675 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:57.844303 (+ 1628us) server_negotiation.cc:407] Connection header received
0807 15:13:57.845272 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:57.845276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:57.845330 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:57.845419 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:57.847213 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:57.847763 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:57.849767 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:57.850007 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:57.850098 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:57.853455 (+ 3357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:57.853458 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:57.853460 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:57.853488 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:57.855154 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:57.855769 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:57.855772 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:57.855774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:57.855864 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:57.856257 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:57.856260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:57.856263 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:57.856470 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:57.856583 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:57.857151 (+ 568us) server_negotiation.cc:299] Negotiation successful
0807 15:13:57.857295 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 15:13:57.858386 12057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:57.842991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:33279 (local address 127.15.226.65:52367)
0807 15:13:57.844111 (+ 1120us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:57.844161 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:57.845046 (+ 885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:57.845636 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:57.845643 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:57.846233 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:57.847015 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:57.847040 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:57.847923 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:57.847929 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:57.849625 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:57.849638 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:57.850214 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:57.850221 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:57.850486 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:57.851259 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:13:57.851286 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:57.853230 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:57.855309 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:57.855316 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:57.855343 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:57.855638 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:57.855969 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:57.855972 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:57.855974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:57.856141 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:57.856585 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:57.856596 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:57.856889 (+ 293us) client_negotiation.cc:769] Sending connection context
0807 15:13:57.857087 (+ 198us) client_negotiation.cc:240] Negotiation successful
0807 15:13:57.857475 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"thread_start_us":697,"threads_started":1}
I20260807 15:13:57.859869 12053 heartbeater.cc:347] Connected to a master server at 127.15.226.126:33279
I20260807 15:13:57.860126 12053 heartbeater.cc:464] Registering TS with master...
I20260807 15:13:57.860832 12053 heartbeater.cc:511] Master 127.15.226.126:33279 requested a full tablet report, sending...
I20260807 15:13:57.862715 11776 ts_manager.cc:194] Registered new tserver with Master: b7b4cbfcbff945f6a6df450584bd9a0b (127.15.226.65:42167)
I20260807 15:13:57.864478 11776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.65@KRBTEST.COM'} at 127.15.226.65:52367
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
Entry for principal kudu/127.15.226.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab.
I20260807 15:13:57.878448 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:33279
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:34173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:13:58.000809 12061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:58.001132 12061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:58.001180 12061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:58.005498 12061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:58.005594 12061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:58.005658 12061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:13:58.011874 12061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:34173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.66.keytab
--principal=kudu/127.15.226.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:33279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:58.013320 12061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:58.014698 12061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:58.025602 12067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:58.025602 12070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:58.025604 12068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:58.026026 12061 server_base.cc:1034] running on GCE node
I20260807 15:13:58.026501 12061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:58.027283 12061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:58.028503 12061 hybrid_clock.cc:648] HybridClock initialized: now 1786115638028466 us; error 52 us; skew 500 ppm
Aug 07 15:13:58 dist-test-slave-3kk6 krb5kdc[11701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115638, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:58.031977 12061 init.cc:377] Logged in from keytab as kudu/127.15.226.66@KRBTEST.COM (short username kudu)
I20260807 15:13:58.033010 12061 webserver.cc:460] Webserver started at http://127.15.226.66:42251/ using document root <none> and password file <none>
I20260807 15:13:58.033833 12061 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:58.033928 12061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:58.034181 12061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:13:58.036651 12061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "47fed0891cb844f9a6680a7c9911a8ac"
format_stamp: "Formatted at 2026-08-07 15:13:58 on dist-test-slave-3kk6"
server_key: "5fe9757f0e426c83d4b558fb75c968ea"
server_key_iv: "f90215400c35351e200675b2649dd037"
server_key_version: "encryptionkey@0"
I20260807 15:13:58.037286 12061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "47fed0891cb844f9a6680a7c9911a8ac"
format_stamp: "Formatted at 2026-08-07 15:13:58 on dist-test-slave-3kk6"
server_key: "5fe9757f0e426c83d4b558fb75c968ea"
server_key_iv: "f90215400c35351e200675b2649dd037"
server_key_version: "encryptionkey@0"
I20260807 15:13:58.041759 12061 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260807 15:13:58.044935 12080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:58.046496 12061 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 15:13:58.046693 12061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "47fed0891cb844f9a6680a7c9911a8ac"
format_stamp: "Formatted at 2026-08-07 15:13:58 on dist-test-slave-3kk6"
server_key: "5fe9757f0e426c83d4b558fb75c968ea"
server_key_iv: "f90215400c35351e200675b2649dd037"
server_key_version: "encryptionkey@0"
I20260807 15:13:58.046856 12061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:58.061864 12061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:58.065929 12061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:58.066205 12061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:58.067031 12061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:58.068027 12061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:58.068110 12061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:58.068192 12061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:58.068254 12061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:58.079150 12061 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:43133
I20260807 15:13:58.079172 12240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:43133 every 8 connection(s)
I20260807 15:13:58.080225 12061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:13:58.087275 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 12061
I20260807 15:13:58.087445 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:13:58.087771 16265 external_mini_cluster.cc:1428] Setting key 75c35f55246846a9fe9f72d15fe342c0
Aug 07 15:13:58 dist-test-slave-3kk6 krb5kdc[11701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115638, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.66@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:13:58.098167 11829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:58.082284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:53615 (local address 127.15.226.126:33279)
0807 15:13:58.082458 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:58.082462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:58.083497 (+ 1035us) server_negotiation.cc:407] Connection header received
0807 15:13:58.085102 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:58.085106 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:58.085178 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:58.085277 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:58.086577 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.087212 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.088681 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.089020 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.089158 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:58.092695 (+ 3537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:58.092699 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:58.092701 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:58.092732 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:58.094606 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:58.095639 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:58.095647 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:58.095653 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:58.095774 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:58.096366 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:58.096373 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:58.096375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:58.096688 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:58.096775 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:58.097712 (+ 937us) server_negotiation.cc:299] Negotiation successful
0807 15:13:58.097913 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 15:13:58.100397 12244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:58.082786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:33279 (local address 127.15.226.66:53615)
0807 15:13:58.083316 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:58.083367 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:58.084778 (+ 1411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:58.085479 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:58.085487 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:58.085882 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:58.086404 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:58.086419 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.087364 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.087367 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:58.088533 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:58.088544 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.089287 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.089292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:58.089484 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:58.090217 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:13:58.090238 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:58.092411 (+ 2173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:58.094865 (+ 2454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:58.094873 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:58.094897 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:58.095458 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:58.095940 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:58.095946 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:58.095950 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:58.096197 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:58.096882 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:58.096890 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:58.097319 (+ 429us) client_negotiation.cc:769] Sending connection context
0807 15:13:58.098424 (+ 1105us) client_negotiation.cc:240] Negotiation successful
0807 15:13:58.098962 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":142,"threads_started":1}
WARNING: no policy specified for kudu/127.15.226.67@KRBTEST.COM; defaulting to no policy
I20260807 15:13:58.103173 12241 heartbeater.cc:347] Connected to a master server at 127.15.226.126:33279
I20260807 15:13:58.103411 12241 heartbeater.cc:464] Registering TS with master...
I20260807 15:13:58.104136 12241 heartbeater.cc:511] Master 127.15.226.126:33279 requested a full tablet report, sending...
I20260807 15:13:58.107419 11776 ts_manager.cc:194] Registered new tserver with Master: 47fed0891cb844f9a6680a7c9911a8ac (127.15.226.66:43133)
I20260807 15:13:58.108304 11776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.66@KRBTEST.COM'} at 127.15.226.66:53615
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
Entry for principal kudu/127.15.226.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab.
I20260807 15:13:58.138109 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:33279
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:34173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:13:58.263108 12248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:58.263419 12248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:58.263476 12248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:58.267589 12248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:58.267663 12248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:58.267726 12248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:13:58.273294 12248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:34173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.67.keytab
--principal=kudu/127.15.226.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:33279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:58.274514 12248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:58.275687 12248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:58.286216 12259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:58.286244 12255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:58.286477 12256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:58.286713 12248 server_base.cc:1034] running on GCE node
I20260807 15:13:58.287184 12248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:58.287936 12248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:58.289119 12248 hybrid_clock.cc:648] HybridClock initialized: now 1786115638289096 us; error 39 us; skew 500 ppm
Aug 07 15:13:58 dist-test-slave-3kk6 krb5kdc[11701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115638, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:13:58.292616 12248 init.cc:377] Logged in from keytab as kudu/127.15.226.67@KRBTEST.COM (short username kudu)
I20260807 15:13:58.293685 12248 webserver.cc:460] Webserver started at http://127.15.226.67:35559/ using document root <none> and password file <none>
I20260807 15:13:58.294397 12248 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:58.294471 12248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:58.294803 12248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:13:58.297305 12248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "0297318371534f98ad963708d9f5bcb7"
format_stamp: "Formatted at 2026-08-07 15:13:58 on dist-test-slave-3kk6"
server_key: "7f94b745979a37da26bb6e11c2a98366"
server_key_iv: "2d64e8dfca40eae462957670fa730949"
server_key_version: "encryptionkey@0"
I20260807 15:13:58.298000 12248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "0297318371534f98ad963708d9f5bcb7"
format_stamp: "Formatted at 2026-08-07 15:13:58 on dist-test-slave-3kk6"
server_key: "7f94b745979a37da26bb6e11c2a98366"
server_key_iv: "2d64e8dfca40eae462957670fa730949"
server_key_version: "encryptionkey@0"
I20260807 15:13:58.302767 12248 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260807 15:13:58.305514 12269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:58.307190 12248 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 15:13:58.307303 12248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "0297318371534f98ad963708d9f5bcb7"
format_stamp: "Formatted at 2026-08-07 15:13:58 on dist-test-slave-3kk6"
server_key: "7f94b745979a37da26bb6e11c2a98366"
server_key_iv: "2d64e8dfca40eae462957670fa730949"
server_key_version: "encryptionkey@0"
I20260807 15:13:58.307407 12248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:58.337378 12248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:58.341151 12248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:58.341334 12248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:58.342042 12248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:58.343048 12248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:58.343101 12248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:58.343173 12248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:58.343207 12248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:58.353029 12248 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:38361
I20260807 15:13:58.353061 12423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:38361 every 8 connection(s)
I20260807 15:13:58.354028 12248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:13:58.357599 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 12248
I20260807 15:13:58.357753 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:13:58.358054 16265 external_mini_cluster.cc:1428] Setting key 55be9d6fbdb01df00c91443be883a94c
Aug 07 15:13:58 dist-test-slave-3kk6 krb5kdc[11701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115638, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.67@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:13:58.369588 11829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:58.356124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:50149 (local address 127.15.226.126:33279)
0807 15:13:58.356314 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:58.356317 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:58.357192 (+ 875us) server_negotiation.cc:407] Connection header received
0807 15:13:58.358946 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:58.358950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:58.359014 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:58.359126 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:58.360903 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.361547 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.362985 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.363206 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.363293 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:58.365998 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:58.366002 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:58.366004 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:58.366030 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:58.367483 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:58.368041 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:58.368043 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:58.368044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:58.368124 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:58.368474 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:58.368477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:58.368478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:58.368667 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:58.368812 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:58.369327 (+ 515us) server_negotiation.cc:299] Negotiation successful
0807 15:13:58.369454 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 15:13:58.370491 12429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:58.356493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:33279 (local address 127.15.226.67:50149)
0807 15:13:58.357007 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:58.357056 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:58.358703 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:58.359353 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:58.359362 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:58.359935 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:58.360708 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:58.360728 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.361716 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.361719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:58.362859 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:58.362871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.363370 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.363374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:58.363528 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:58.364166 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:13:58.364184 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:58.365813 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:58.367607 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:58.367613 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:58.367628 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:58.367944 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:58.368208 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:13:58.368210 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:13:58.368212 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:13:58.368380 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:13:58.368804 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:58.368811 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:58.369086 (+ 275us) client_negotiation.cc:769] Sending connection context
0807 15:13:58.369261 (+ 175us) client_negotiation.cc:240] Negotiation successful
0807 15:13:58.369570 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":127,"threads_started":1}
I20260807 15:13:58.371994 12424 heartbeater.cc:347] Connected to a master server at 127.15.226.126:33279
I20260807 15:13:58.372270 12424 heartbeater.cc:464] Registering TS with master...
I20260807 15:13:58.372838 12424 heartbeater.cc:511] Master 127.15.226.126:33279 requested a full tablet report, sending...
I20260807 15:13:58.373999 11776 ts_manager.cc:194] Registered new tserver with Master: 0297318371534f98ad963708d9f5bcb7 (127.15.226.67:38361)
I20260807 15:13:58.374629 11776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.226.67@KRBTEST.COM'} at 127.15.226.67:50149
I20260807 15:13:58.385042 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:13:58.394852 11829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:58.387518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43642 (local address 127.15.226.126:33279)
0807 15:13:58.387653 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:58.387658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:58.387682 (+ 24us) server_negotiation.cc:407] Connection header received
0807 15:13:58.387793 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:58.387795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:58.387838 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:58.387910 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:58.388670 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.389209 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.390051 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.390262 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.390343 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:58.391882 (+ 1539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:58.391885 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:58.391887 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:13:58.391911 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:58.393238 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:58.393640 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:58.393643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:58.393644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:58.393723 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:13:58.393982 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:13:58.393984 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:13:58.393985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:13:58.394214 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:58.394322 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:58.394528 (+ 206us) server_negotiation.cc:299] Negotiation successful
0807 15:13:58.394636 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 15:13:58.398063 11776 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43642:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:13:58.401297 11776 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:13:58.417884 12447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:58.411737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38490 (local address 127.15.226.65:42167)
0807 15:13:58.412637 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:58.412644 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:58.412674 (+ 30us) server_negotiation.cc:407] Connection header received
0807 15:13:58.412783 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:58.412788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:58.413071 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:58.413226 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:58.413487 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.414463 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.416471 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.417091 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.417233 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:58.417417 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:58.417570 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 15:13:58.417681 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":100,"threads_started":1}
I20260807 15:13:58.417857 12441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:58.410890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:42167 (local address 127.0.0.1:38490)
0807 15:13:58.411994 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:58.412009 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:58.412152 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:58.413199 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:58.413202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:58.413221 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:58.413347 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:58.413354 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.414583 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.414588 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:58.416325 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:58.416333 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.417237 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.417241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:58.417357 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:58.417396 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 15:13:58.417527 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 15:13:58.417673 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1033,"mutex_wait_us":133,"thread_start_us":97,"threads_started":1}
I20260807 15:13:58.418406 12440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:58.410807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:38361 (local address 127.0.0.1:35624)
0807 15:13:58.411325 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:58.411389 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:58.411537 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:58.412759 (+ 1222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:58.412767 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:58.412794 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:58.412955 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:58.412962 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.414894 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.414898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:58.416295 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:58.416308 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.417543 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.417547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:58.417639 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:58.417662 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 15:13:58.417735 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 15:13:58.417812 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":73,"thread_start_us":127,"threads_started":1}
I20260807 15:13:58.418813 12444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:58.411143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35624 (local address 127.15.226.67:38361)
0807 15:13:58.411751 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:58.411767 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:58.411793 (+ 26us) server_negotiation.cc:407] Connection header received
0807 15:13:58.412181 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:58.412191 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:58.412419 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:58.413163 (+ 744us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:58.413336 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.414638 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.416443 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.417387 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.417936 (+ 549us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:58.418131 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:58.418183 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 15:13:58.418278 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":123,"threads_started":1}
I20260807 15:13:58.419116 12448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:58.412224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43958 (local address 127.15.226.66:43133)
0807 15:13:58.412697 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:58.412711 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:58.412734 (+ 23us) server_negotiation.cc:407] Connection header received
0807 15:13:58.412808 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:58.412813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:58.412990 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:58.413135 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:58.413664 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.414817 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.416485 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.417387 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.417514 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:58.417661 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:58.418682 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0807 15:13:58.418800 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":170,"threads_started":1}
I20260807 15:13:58.419754 12446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:58.411680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:43133 (local address 127.0.0.1:43958)
0807 15:13:58.412143 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:58.412157 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:58.412307 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:58.413250 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:58.413253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:58.413271 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:58.413521 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:58.413532 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.415121 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.415132 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:58.416313 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:58.416325 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.417727 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.417733 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:58.417810 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:58.417830 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 15:13:58.417898 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 15:13:58.418343 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":199,"thread_start_us":104,"threads_started":1}
I20260807 15:13:58.421566 12334 tablet_service.cc:1467] Processing CreateTablet for tablet e3dfb3454b594822a4d767e54a751941 (DEFAULT_TABLE table=test-table [id=684b0d30381f49188d432c8830c9e2d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:13:58.421578 11961 tablet_service.cc:1467] Processing CreateTablet for tablet e3dfb3454b594822a4d767e54a751941 (DEFAULT_TABLE table=test-table [id=684b0d30381f49188d432c8830c9e2d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:13:58.422168 12148 tablet_service.cc:1467] Processing CreateTablet for tablet e3dfb3454b594822a4d767e54a751941 (DEFAULT_TABLE table=test-table [id=684b0d30381f49188d432c8830c9e2d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:13:58.422907 11961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3dfb3454b594822a4d767e54a751941. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:58.422906 12334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3dfb3454b594822a4d767e54a751941. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:58.423282 12148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3dfb3454b594822a4d767e54a751941. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:58.430004 12450 tablet_bootstrap.cc:492] T e3dfb3454b594822a4d767e54a751941 P 47fed0891cb844f9a6680a7c9911a8ac: Bootstrap starting.
I20260807 15:13:58.431435 12451 tablet_bootstrap.cc:492] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b: Bootstrap starting.
I20260807 15:13:58.432531 12450 tablet_bootstrap.cc:654] T e3dfb3454b594822a4d767e54a751941 P 47fed0891cb844f9a6680a7c9911a8ac: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:58.433342 12451 tablet_bootstrap.cc:654] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:58.433871 12452 tablet_bootstrap.cc:492] T e3dfb3454b594822a4d767e54a751941 P 0297318371534f98ad963708d9f5bcb7: Bootstrap starting.
I20260807 15:13:58.434253 12451 log.cc:824] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:58.434271 12450 log.cc:824] T e3dfb3454b594822a4d767e54a751941 P 47fed0891cb844f9a6680a7c9911a8ac: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:58.436117 12452 tablet_bootstrap.cc:654] T e3dfb3454b594822a4d767e54a751941 P 0297318371534f98ad963708d9f5bcb7: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:58.436625 12450 tablet_bootstrap.cc:492] T e3dfb3454b594822a4d767e54a751941 P 47fed0891cb844f9a6680a7c9911a8ac: No bootstrap required, opened a new log
I20260807 15:13:58.436625 12451 tablet_bootstrap.cc:492] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b: No bootstrap required, opened a new log
I20260807 15:13:58.436842 12451 ts_tablet_manager.cc:1397] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.001s
I20260807 15:13:58.436842 12450 ts_tablet_manager.cc:1397] T e3dfb3454b594822a4d767e54a751941 P 47fed0891cb844f9a6680a7c9911a8ac: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20260807 15:13:58.437359 12452 log.cc:824] T e3dfb3454b594822a4d767e54a751941 P 0297318371534f98ad963708d9f5bcb7: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:58.439251 12452 tablet_bootstrap.cc:492] T e3dfb3454b594822a4d767e54a751941 P 0297318371534f98ad963708d9f5bcb7: No bootstrap required, opened a new log
I20260807 15:13:58.439473 12452 ts_tablet_manager.cc:1397] T e3dfb3454b594822a4d767e54a751941 P 0297318371534f98ad963708d9f5bcb7: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 15:13:58.441249 12450 raft_consensus.cc:348] T e3dfb3454b594822a4d767e54a751941 P 47fed0891cb844f9a6680a7c9911a8ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0297318371534f98ad963708d9f5bcb7" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 38361 } } peers { permanent_uuid: "b7b4cbfcbff945f6a6df450584bd9a0b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42167 } } peers { permanent_uuid: "47fed0891cb844f9a6680a7c9911a8ac" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43133 } }
I20260807 15:13:58.441583 12450 raft_consensus.cc:374] T e3dfb3454b594822a4d767e54a751941 P 47fed0891cb844f9a6680a7c9911a8ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:58.441639 12450 raft_consensus.cc:729] T e3dfb3454b594822a4d767e54a751941 P 47fed0891cb844f9a6680a7c9911a8ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47fed0891cb844f9a6680a7c9911a8ac, State: Initialized, Role: FOLLOWER
I20260807 15:13:58.441659 12451 raft_consensus.cc:348] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0297318371534f98ad963708d9f5bcb7" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 38361 } } peers { permanent_uuid: "b7b4cbfcbff945f6a6df450584bd9a0b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42167 } } peers { permanent_uuid: "47fed0891cb844f9a6680a7c9911a8ac" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43133 } }
I20260807 15:13:58.441946 12451 raft_consensus.cc:374] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:58.442008 12451 raft_consensus.cc:729] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7b4cbfcbff945f6a6df450584bd9a0b, State: Initialized, Role: FOLLOWER
I20260807 15:13:58.442162 12450 consensus_queue.cc:260] T e3dfb3454b594822a4d767e54a751941 P 47fed0891cb844f9a6680a7c9911a8ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0297318371534f98ad963708d9f5bcb7" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 38361 } } peers { permanent_uuid: "b7b4cbfcbff945f6a6df450584bd9a0b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42167 } } peers { permanent_uuid: "47fed0891cb844f9a6680a7c9911a8ac" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43133 } }
I20260807 15:13:58.442431 12451 consensus_queue.cc:260] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0297318371534f98ad963708d9f5bcb7" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 38361 } } peers { permanent_uuid: "b7b4cbfcbff945f6a6df450584bd9a0b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42167 } } peers { permanent_uuid: "47fed0891cb844f9a6680a7c9911a8ac" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43133 } }
I20260807 15:13:58.443132 12241 heartbeater.cc:503] Master 127.15.226.126:33279 was elected leader, sending a full tablet report...
I20260807 15:13:58.443414 12451 ts_tablet_manager.cc:1428] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b: Time spent starting tablet: real 0.007s user 0.005s sys 0.002s
I20260807 15:13:58.443414 12450 ts_tablet_manager.cc:1428] T e3dfb3454b594822a4d767e54a751941 P 47fed0891cb844f9a6680a7c9911a8ac: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260807 15:13:58.443270 12452 raft_consensus.cc:348] T e3dfb3454b594822a4d767e54a751941 P 0297318371534f98ad963708d9f5bcb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0297318371534f98ad963708d9f5bcb7" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 38361 } } peers { permanent_uuid: "b7b4cbfcbff945f6a6df450584bd9a0b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42167 } } peers { permanent_uuid: "47fed0891cb844f9a6680a7c9911a8ac" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43133 } }
I20260807 15:13:58.443519 12452 raft_consensus.cc:374] T e3dfb3454b594822a4d767e54a751941 P 0297318371534f98ad963708d9f5bcb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:58.443567 12452 raft_consensus.cc:729] T e3dfb3454b594822a4d767e54a751941 P 0297318371534f98ad963708d9f5bcb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0297318371534f98ad963708d9f5bcb7, State: Initialized, Role: FOLLOWER
I20260807 15:13:58.443790 12053 heartbeater.cc:503] Master 127.15.226.126:33279 was elected leader, sending a full tablet report...
I20260807 15:13:58.443979 12452 consensus_queue.cc:260] T e3dfb3454b594822a4d767e54a751941 P 0297318371534f98ad963708d9f5bcb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0297318371534f98ad963708d9f5bcb7" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 38361 } } peers { permanent_uuid: "b7b4cbfcbff945f6a6df450584bd9a0b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42167 } } peers { permanent_uuid: "47fed0891cb844f9a6680a7c9911a8ac" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43133 } }
I20260807 15:13:58.444766 12424 heartbeater.cc:503] Master 127.15.226.126:33279 was elected leader, sending a full tablet report...
I20260807 15:13:58.445904 12452 ts_tablet_manager.cc:1428] T e3dfb3454b594822a4d767e54a751941 P 0297318371534f98ad963708d9f5bcb7: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 15:13:58.512064 12458 raft_consensus.cc:482] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:13:58.512317 12458 raft_consensus.cc:504] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0297318371534f98ad963708d9f5bcb7" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 38361 } } peers { permanent_uuid: "b7b4cbfcbff945f6a6df450584bd9a0b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42167 } } peers { permanent_uuid: "47fed0891cb844f9a6680a7c9911a8ac" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43133 } }
I20260807 15:13:58.513554 12458 leader_election.cc:288] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0297318371534f98ad963708d9f5bcb7 (127.15.226.67:38361), 47fed0891cb844f9a6680a7c9911a8ac (127.15.226.66:43133)
I20260807 15:13:58.519281 12444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:58.514023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:48807 (local address 127.15.226.67:38361)
0807 15:13:58.514203 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:58.514209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:58.514441 (+ 232us) server_negotiation.cc:407] Connection header received
0807 15:13:58.514603 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:58.514606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:58.514715 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:58.514795 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:58.515173 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.515993 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.517853 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.518513 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.518681 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:58.518746 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:58.519046 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 15:13:58.519140 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 15:13:58.519281 12448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:58.514538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:35801 (local address 127.15.226.66:43133)
0807 15:13:58.514709 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:58.514714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:58.514920 (+ 206us) server_negotiation.cc:407] Connection header received
0807 15:13:58.515033 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:58.515035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:58.515078 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:58.515183 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:58.515440 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.516152 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.517845 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.518513 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.518680 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:58.518746 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:58.519046 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 15:13:58.519140 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:13:58.519330 12462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:58.514380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:43133 (local address 127.15.226.65:35801)
0807 15:13:58.514835 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:58.514849 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:58.514950 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:58.515210 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:58.515212 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:58.515228 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:58.515322 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:58.515326 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.516252 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.516255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:58.517637 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:58.517651 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.518686 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.518690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:58.518842 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:58.518890 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 15:13:58.518981 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 15:13:58.519095 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":71,"thread_start_us":114,"threads_started":1}
I20260807 15:13:58.519335 12460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:58.513885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:38361 (local address 127.15.226.65:48807)
0807 15:13:58.514297 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:58.514313 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:58.514483 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:58.514835 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:58.514839 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:58.514866 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:58.515041 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:58.515049 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.516111 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.516114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:58.517637 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:58.517651 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.518686 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.518690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:58.518845 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:58.518894 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 15:13:58.518980 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 15:13:58.519095 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":179,"thread_start_us":170,"threads_started":1}
I20260807 15:13:58.520170 12175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3dfb3454b594822a4d767e54a751941" candidate_uuid: "b7b4cbfcbff945f6a6df450584bd9a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47fed0891cb844f9a6680a7c9911a8ac" is_pre_election: true
I20260807 15:13:58.520511 12360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3dfb3454b594822a4d767e54a751941" candidate_uuid: "b7b4cbfcbff945f6a6df450584bd9a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0297318371534f98ad963708d9f5bcb7" is_pre_election: true
I20260807 15:13:58.520545 12175 raft_consensus.cc:2454] T e3dfb3454b594822a4d767e54a751941 P 47fed0891cb844f9a6680a7c9911a8ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7b4cbfcbff945f6a6df450584bd9a0b in term 0.
I20260807 15:13:58.520820 12360 raft_consensus.cc:2454] T e3dfb3454b594822a4d767e54a751941 P 0297318371534f98ad963708d9f5bcb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7b4cbfcbff945f6a6df450584bd9a0b in term 0.
I20260807 15:13:58.521162 11894 leader_election.cc:302] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47fed0891cb844f9a6680a7c9911a8ac, b7b4cbfcbff945f6a6df450584bd9a0b; no voters:
I20260807 15:13:58.521474 12458 raft_consensus.cc:2781] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:13:58.521577 12458 raft_consensus.cc:482] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:13:58.521634 12458 raft_consensus.cc:3037] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:58.522953 12458 raft_consensus.cc:504] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0297318371534f98ad963708d9f5bcb7" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 38361 } } peers { permanent_uuid: "b7b4cbfcbff945f6a6df450584bd9a0b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42167 } } peers { permanent_uuid: "47fed0891cb844f9a6680a7c9911a8ac" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43133 } }
I20260807 15:13:58.523407 12458 leader_election.cc:288] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b [CANDIDATE]: Term 1 election: Requested vote from peers 0297318371534f98ad963708d9f5bcb7 (127.15.226.67:38361), 47fed0891cb844f9a6680a7c9911a8ac (127.15.226.66:43133)
I20260807 15:13:58.523906 12360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3dfb3454b594822a4d767e54a751941" candidate_uuid: "b7b4cbfcbff945f6a6df450584bd9a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0297318371534f98ad963708d9f5bcb7"
I20260807 15:13:58.523977 12175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3dfb3454b594822a4d767e54a751941" candidate_uuid: "b7b4cbfcbff945f6a6df450584bd9a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47fed0891cb844f9a6680a7c9911a8ac"
I20260807 15:13:58.524042 12360 raft_consensus.cc:3037] T e3dfb3454b594822a4d767e54a751941 P 0297318371534f98ad963708d9f5bcb7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:58.524155 12175 raft_consensus.cc:3037] T e3dfb3454b594822a4d767e54a751941 P 47fed0891cb844f9a6680a7c9911a8ac [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:58.525368 12360 raft_consensus.cc:2454] T e3dfb3454b594822a4d767e54a751941 P 0297318371534f98ad963708d9f5bcb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7b4cbfcbff945f6a6df450584bd9a0b in term 1.
I20260807 15:13:58.525369 12175 raft_consensus.cc:2454] T e3dfb3454b594822a4d767e54a751941 P 47fed0891cb844f9a6680a7c9911a8ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7b4cbfcbff945f6a6df450584bd9a0b in term 1.
I20260807 15:13:58.525760 11894 leader_election.cc:302] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0297318371534f98ad963708d9f5bcb7, b7b4cbfcbff945f6a6df450584bd9a0b; no voters:
I20260807 15:13:58.525969 12458 raft_consensus.cc:2781] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:13:58.526232 12458 raft_consensus.cc:686] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b [term 1 LEADER]: Becoming Leader. State: Replica: b7b4cbfcbff945f6a6df450584bd9a0b, State: Running, Role: LEADER
I20260807 15:13:58.526643 12458 consensus_queue.cc:237] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0297318371534f98ad963708d9f5bcb7" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 38361 } } peers { permanent_uuid: "b7b4cbfcbff945f6a6df450584bd9a0b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42167 } } peers { permanent_uuid: "47fed0891cb844f9a6680a7c9911a8ac" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43133 } }
I20260807 15:13:58.530297 11772 catalog_manager.cc:5495] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b reported cstate change: term changed from 0 to 1, leader changed from <none> to b7b4cbfcbff945f6a6df450584bd9a0b (127.15.226.65). New cstate: current_term: 1 leader_uuid: "b7b4cbfcbff945f6a6df450584bd9a0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0297318371534f98ad963708d9f5bcb7" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 38361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7b4cbfcbff945f6a6df450584bd9a0b" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 42167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "47fed0891cb844f9a6680a7c9911a8ac" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43133 } health_report { overall_health: UNKNOWN } } }
I20260807 15:13:58.544965 12447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:58.540661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38500 (local address 127.15.226.65:42167)
0807 15:13:58.540825 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:58.540830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:58.540848 (+ 18us) server_negotiation.cc:407] Connection header received
0807 15:13:58.541011 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:58.541014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:58.541085 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:58.541196 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:13:58.541433 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.542246 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.543242 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.543603 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.543712 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:58.544017 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:13:58.544562 (+ 545us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:13:58.544687 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:58.544749 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 15:13:58.544812 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 15:13:58.551433 12360 raft_consensus.cc:1261] T e3dfb3454b594822a4d767e54a751941 P 0297318371534f98ad963708d9f5bcb7 [term 1 FOLLOWER]: Refusing update from remote peer b7b4cbfcbff945f6a6df450584bd9a0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:13:58.551472 12175 raft_consensus.cc:1261] T e3dfb3454b594822a4d767e54a751941 P 47fed0891cb844f9a6680a7c9911a8ac [term 1 FOLLOWER]: Refusing update from remote peer b7b4cbfcbff945f6a6df450584bd9a0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:13:58.552235 12458 consensus_queue.cc:1035] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0297318371534f98ad963708d9f5bcb7" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 38361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:13:58.552513 12463 consensus_queue.cc:1035] T e3dfb3454b594822a4d767e54a751941 P b7b4cbfcbff945f6a6df450584bd9a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "47fed0891cb844f9a6680a7c9911a8ac" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 43133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:13:58.571321 11772 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:13:58.571506 11772 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43642:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:13:58.574198 11772 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6326ba4bf28248399f9f193a781a1391: Sending DeleteTablet for 3 replicas of tablet e3dfb3454b594822a4d767e54a751941
I20260807 15:13:58.575167 11961 tablet_service.cc:1514] Processing DeleteTablet for tablet e3dfb3454b594822a4d767e54a751941 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:13:58 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:38490
I20260807 15:13:58.575253 12148 tablet_service.cc:1514] Processing DeleteTablet for tablet e3dfb3454b594822a4d767e54a751941 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:13:58 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:43958
I20260807 15:13:58.575834 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 11866
I20260807 15:13:58.576013 12334 tablet_service.cc:1514] Processing DeleteTablet for tablet e3dfb3454b594822a4d767e54a751941 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:13:58 UTC) from {username='kudu', principal='kudu/127.15.226.126@KRBTEST.COM'} at 127.0.0.1:35624
I20260807 15:13:58.580134 12483 tablet_replica.cc:330] T e3dfb3454b594822a4d767e54a751941 P 47fed0891cb844f9a6680a7c9911a8ac: stopping tablet replica
I20260807 15:13:58.580782 12483 raft_consensus.cc:2229] T e3dfb3454b594822a4d767e54a751941 P 47fed0891cb844f9a6680a7c9911a8ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:13:58.581331 12483 raft_consensus.cc:2258] T e3dfb3454b594822a4d767e54a751941 P 47fed0891cb844f9a6680a7c9911a8ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:13:58.585673 12484 tablet_replica.cc:330] T e3dfb3454b594822a4d767e54a751941 P 0297318371534f98ad963708d9f5bcb7: stopping tablet replica
I20260807 15:13:58.586145 12484 raft_consensus.cc:2229] T e3dfb3454b594822a4d767e54a751941 P 0297318371534f98ad963708d9f5bcb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:13:58.586555 12484 raft_consensus.cc:2258] T e3dfb3454b594822a4d767e54a751941 P 0297318371534f98ad963708d9f5bcb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:13:58.586970 12483 ts_tablet_manager.cc:1905] T e3dfb3454b594822a4d767e54a751941 P 47fed0891cb844f9a6680a7c9911a8ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:13:58.588140 12484 ts_tablet_manager.cc:1905] T e3dfb3454b594822a4d767e54a751941 P 0297318371534f98ad963708d9f5bcb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:13:58.590310 12483 ts_tablet_manager.cc:1918] T e3dfb3454b594822a4d767e54a751941 P 47fed0891cb844f9a6680a7c9911a8ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:13:58.590448 12483 log.cc:1196] T e3dfb3454b594822a4d767e54a751941 P 47fed0891cb844f9a6680a7c9911a8ac: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/e3dfb3454b594822a4d767e54a751941
I20260807 15:13:58.590842 12483 ts_tablet_manager.cc:1939] T e3dfb3454b594822a4d767e54a751941 P 47fed0891cb844f9a6680a7c9911a8ac: Deleting consensus metadata
I20260807 15:13:58.591408 12484 ts_tablet_manager.cc:1918] T e3dfb3454b594822a4d767e54a751941 P 0297318371534f98ad963708d9f5bcb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:13:58.591524 12484 log.cc:1196] T e3dfb3454b594822a4d767e54a751941 P 0297318371534f98ad963708d9f5bcb7: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/e3dfb3454b594822a4d767e54a751941
I20260807 15:13:58.591953 12484 ts_tablet_manager.cc:1939] T e3dfb3454b594822a4d767e54a751941 P 0297318371534f98ad963708d9f5bcb7: Deleting consensus metadata
I20260807 15:13:58.592634 11752 catalog_manager.cc:4841] TS 47fed0891cb844f9a6680a7c9911a8ac (127.15.226.66:43133): tablet e3dfb3454b594822a4d767e54a751941 (table test-table [id=684b0d30381f49188d432c8830c9e2d6]) successfully deleted
I20260807 15:13:58.593292 11752 catalog_manager.cc:4841] TS 0297318371534f98ad963708d9f5bcb7 (127.15.226.67:38361): tablet e3dfb3454b594822a4d767e54a751941 (table test-table [id=684b0d30381f49188d432c8830c9e2d6]) successfully deleted
W20260807 15:13:58.593365 11749 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.226.65:42167 (error 108)
I20260807 15:13:58.593595 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 12061
I20260807 15:13:58.597970 12446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:58.596705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:42167 (local address 127.0.0.1:38506)
0807 15:13:58.597747 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:58.597847 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.65:42167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":920}
W20260807 15:13:58.598189 11749 catalog_manager.cc:4584] TS b7b4cbfcbff945f6a6df450584bd9a0b (127.15.226.65:42167): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e3dfb3454b594822a4d767e54a751941: Network error: Client connection negotiation failed: client connection to 127.15.226.65:42167: connect: Connection refused (error 111)
I20260807 15:13:58.612143 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 12248
I20260807 15:13:58.627826 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 11720
2026-08-07T15:13:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2680 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T15:13:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:13:58Z Disabled control of system clock
I20260807 15:13:58.669899 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:36843
--webserver_interface=127.15.226.126
--webserver_port=0
--builtin_ntp_servers=127.15.226.84:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:36843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:13:58.788375 12489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:58.788682 12489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:58.788720 12489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:58.792764 12489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:13:58.792845 12489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:58.792862 12489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:58.792879 12489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:13:58.792896 12489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:13:58.797878 12489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:36843
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:36843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:58.799018 12489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:58.800416 12489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:58.809307 12495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:58.809324 12499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:58.809403 12494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:58.809724 12489 server_base.cc:1034] running on GCE node
I20260807 15:13:58.810256 12489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:58.811494 12489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:58.812670 12489 hybrid_clock.cc:648] HybridClock initialized: now 1786115638812656 us; error 44 us; skew 500 ppm
I20260807 15:13:58.814721 12489 webserver.cc:460] Webserver started at http://127.15.226.126:40937/ using document root <none> and password file <none>
I20260807 15:13:58.815299 12489 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:58.815372 12489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:58.815548 12489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:13:58.817764 12489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "86ace2792b2f44ae99bcf9ec4f652641"
format_stamp: "Formatted at 2026-08-07 15:13:58 on dist-test-slave-3kk6"
server_key: "454ca72bec2cb4dfa612de29cd467cd5"
server_key_iv: "3b5c738186a799bacbff1dfd18a65754"
server_key_version: "encryptionkey@0"
I20260807 15:13:58.818306 12489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "86ace2792b2f44ae99bcf9ec4f652641"
format_stamp: "Formatted at 2026-08-07 15:13:58 on dist-test-slave-3kk6"
server_key: "454ca72bec2cb4dfa612de29cd467cd5"
server_key_iv: "3b5c738186a799bacbff1dfd18a65754"
server_key_version: "encryptionkey@0"
I20260807 15:13:58.822134 12489 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:13:58.824769 12506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:58.826018 12489 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 15:13:58.826157 12489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "86ace2792b2f44ae99bcf9ec4f652641"
format_stamp: "Formatted at 2026-08-07 15:13:58 on dist-test-slave-3kk6"
server_key: "454ca72bec2cb4dfa612de29cd467cd5"
server_key_iv: "3b5c738186a799bacbff1dfd18a65754"
server_key_version: "encryptionkey@0"
I20260807 15:13:58.826277 12489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:58.865890 12489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:58.866748 12489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:58.866990 12489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:58.874722 12489 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:36843
I20260807 15:13:58.874799 12589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:36843 every 8 connection(s)
I20260807 15:13:58.875780 12489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:13:58.877789 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 12489
I20260807 15:13:58.877924 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:13:58.878232 16265 external_mini_cluster.cc:1428] Setting key 6f668d01c6069ef58c38f403e76c56ff
I20260807 15:13:58.879403 12591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:58.886423 12591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86ace2792b2f44ae99bcf9ec4f652641: Bootstrap starting.
I20260807 15:13:58.888283 12595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:58.879813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45808 (local address 127.15.226.126:36843)
0807 15:13:58.880501 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:58.880517 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:58.880554 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:13:58.881234 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:58.881250 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:58.881662 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:58.881966 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:58.882427 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.883507 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.884739 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:58.885219 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:58.885357 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:58.886317 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:58.886322 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:58.886349 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:13:58.886840 (+ 491us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:58.886922 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:13:58.886944 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:58.887024 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:58.887195 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 15:13:58.887512 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":143,"threads_started":1}
I20260807 15:13:58.889029 12591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86ace2792b2f44ae99bcf9ec4f652641: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:58.889859 12591 log.cc:824] T 00000000000000000000000000000000 P 86ace2792b2f44ae99bcf9ec4f652641: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:58.891986 12591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86ace2792b2f44ae99bcf9ec4f652641: No bootstrap required, opened a new log
I20260807 15:13:58.895714 12591 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86ace2792b2f44ae99bcf9ec4f652641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86ace2792b2f44ae99bcf9ec4f652641" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36843 } }
I20260807 15:13:58.896076 12591 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86ace2792b2f44ae99bcf9ec4f652641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:58.896143 12591 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86ace2792b2f44ae99bcf9ec4f652641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86ace2792b2f44ae99bcf9ec4f652641, State: Initialized, Role: FOLLOWER
I20260807 15:13:58.896693 12591 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86ace2792b2f44ae99bcf9ec4f652641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86ace2792b2f44ae99bcf9ec4f652641" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36843 } }
I20260807 15:13:58.896862 12591 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86ace2792b2f44ae99bcf9ec4f652641 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:13:58.896937 12591 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86ace2792b2f44ae99bcf9ec4f652641 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:13:58.897046 12591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86ace2792b2f44ae99bcf9ec4f652641 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:58.898155 12591 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86ace2792b2f44ae99bcf9ec4f652641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86ace2792b2f44ae99bcf9ec4f652641" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36843 } }
I20260807 15:13:58.898546 12591 leader_election.cc:302] T 00000000000000000000000000000000 P 86ace2792b2f44ae99bcf9ec4f652641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86ace2792b2f44ae99bcf9ec4f652641; no voters:
I20260807 15:13:58.898939 12591 leader_election.cc:288] T 00000000000000000000000000000000 P 86ace2792b2f44ae99bcf9ec4f652641 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:13:58.899137 12597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86ace2792b2f44ae99bcf9ec4f652641 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:13:58.899379 12597 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86ace2792b2f44ae99bcf9ec4f652641 [term 1 LEADER]: Becoming Leader. State: Replica: 86ace2792b2f44ae99bcf9ec4f652641, State: Running, Role: LEADER
I20260807 15:13:58.899706 12597 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86ace2792b2f44ae99bcf9ec4f652641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86ace2792b2f44ae99bcf9ec4f652641" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36843 } }
I20260807 15:13:58.900362 12591 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86ace2792b2f44ae99bcf9ec4f652641 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:13:58.901647 12599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86ace2792b2f44ae99bcf9ec4f652641 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86ace2792b2f44ae99bcf9ec4f652641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86ace2792b2f44ae99bcf9ec4f652641" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36843 } } }
I20260807 15:13:58.901679 12600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86ace2792b2f44ae99bcf9ec4f652641 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86ace2792b2f44ae99bcf9ec4f652641. Latest consensus state: current_term: 1 leader_uuid: "86ace2792b2f44ae99bcf9ec4f652641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86ace2792b2f44ae99bcf9ec4f652641" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36843 } } }
I20260807 15:13:58.901785 12599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86ace2792b2f44ae99bcf9ec4f652641 [sys.catalog]: This master's current role is: LEADER
I20260807 15:13:58.901798 12600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86ace2792b2f44ae99bcf9ec4f652641 [sys.catalog]: This master's current role is: LEADER
I20260807 15:13:58.902173 12609 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:13:58.905333 12609 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:13:58.911636 12609 catalog_manager.cc:1347] Generated new cluster ID: 79b6da530279447f8f8425fa2e091909
I20260807 15:13:58.911746 12609 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:13:58.920972 12609 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:13:58.922312 12609 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:13:58.931479 12609 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86ace2792b2f44ae99bcf9ec4f652641: Generated new TSK 0
I20260807 15:13:58.932379 12609 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:13:58.935632 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.65:0
--local_ip_for_outbound_sockets=127.15.226.65
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:36843
--builtin_ntp_servers=127.15.226.84:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 15:13:59.070696 12621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:59.071058 12621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:59.071134 12621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:59.075544 12621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:59.075644 12621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:59.075743 12621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.65
I20260807 15:13:59.080739 12621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.226.65
--webserver_port=0
--tserver_master_addrs=127.15.226.126:36843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.65
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:59.081959 12621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:59.083581 12621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:59.093865 12632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:59.093866 12630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:59.094146 12621 server_base.cc:1034] running on GCE node
W20260807 15:13:59.093865 12628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:59.094764 12621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:59.095538 12621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:59.096720 12621 hybrid_clock.cc:648] HybridClock initialized: now 1786115639096703 us; error 37 us; skew 500 ppm
I20260807 15:13:59.099025 12621 webserver.cc:460] Webserver started at http://127.15.226.65:45095/ using document root <none> and password file <none>
I20260807 15:13:59.099658 12621 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:59.099731 12621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:59.100040 12621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:13:59.102483 12621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-0/data/instance:
uuid: "f8b539e910be40779693c54e7da371c2"
format_stamp: "Formatted at 2026-08-07 15:13:59 on dist-test-slave-3kk6"
server_key: "1bd14c4bcd29b92d1be68d5229c0c7da"
server_key_iv: "1ebd639c922296cd65a52bffdac12c1b"
server_key_version: "encryptionkey@0"
I20260807 15:13:59.103183 12621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance:
uuid: "f8b539e910be40779693c54e7da371c2"
format_stamp: "Formatted at 2026-08-07 15:13:59 on dist-test-slave-3kk6"
server_key: "1bd14c4bcd29b92d1be68d5229c0c7da"
server_key_iv: "1ebd639c922296cd65a52bffdac12c1b"
server_key_version: "encryptionkey@0"
I20260807 15:13:59.107285 12621 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:13:59.109954 12644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:59.111363 12621 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:13:59.111497 12621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-0/wal
uuid: "f8b539e910be40779693c54e7da371c2"
format_stamp: "Formatted at 2026-08-07 15:13:59 on dist-test-slave-3kk6"
server_key: "1bd14c4bcd29b92d1be68d5229c0c7da"
server_key_iv: "1ebd639c922296cd65a52bffdac12c1b"
server_key_version: "encryptionkey@0"
I20260807 15:13:59.111620 12621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:59.136268 12621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:59.137070 12621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:59.137295 12621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:59.137966 12621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:59.139057 12621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:59.139133 12621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:59.139209 12621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:59.139252 12621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:59.149634 12621 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.65:44729
I20260807 15:13:59.149724 12797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.65:44729 every 8 connection(s)
I20260807 15:13:59.150741 12621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-0/data/info.pb
I20260807 15:13:59.154760 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 12621
I20260807 15:13:59.154937 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-0/wal/instance
I20260807 15:13:59.155284 16265 external_mini_cluster.cc:1428] Setting key 31fb6661e703930731cca77803eaedf0
I20260807 15:13:59.157514 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.66:0
--local_ip_for_outbound_sockets=127.15.226.66
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:36843
--builtin_ntp_servers=127.15.226.84:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 15:13:59.162405 12595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:59.153026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.65:48021 (local address 127.15.226.126:36843)
0807 15:13:59.153196 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:59.153200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:59.154343 (+ 1143us) server_negotiation.cc:407] Connection header received
0807 15:13:59.156063 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:59.156066 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:59.156133 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:59.156237 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:59.157203 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.157848 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.159130 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.159400 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.159524 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:59.161773 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:59.161778 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:59.161784 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:13:59.161803 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:59.161838 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:13:59.161850 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:59.161899 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:59.162157 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 15:13:59.162248 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 15:13:59.163411 12800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:59.153462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36843 (local address 127.15.226.65:48021)
0807 15:13:59.154172 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:59.154215 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:59.155829 (+ 1614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:59.156437 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:59.156444 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:59.156534 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:59.157058 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:59.157071 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.157986 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.157989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:59.158999 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:59.159010 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.159726 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.159731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:59.159984 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:59.161358 (+ 1374us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:13:59.161362 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:59.161436 (+ 74us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:13:59.161549 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:59.161970 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:59.161982 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:59.161988 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:13:59.162152 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 15:13:59.162491 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":132,"threads_started":1}
I20260807 15:13:59.165092 12798 heartbeater.cc:347] Connected to a master server at 127.15.226.126:36843
I20260807 15:13:59.165367 12798 heartbeater.cc:464] Registering TS with master...
I20260807 15:13:59.165997 12798 heartbeater.cc:511] Master 127.15.226.126:36843 requested a full tablet report, sending...
I20260807 15:13:59.168116 12529 ts_manager.cc:194] Registered new tserver with Master: f8b539e910be40779693c54e7da371c2 (127.15.226.65:44729)
I20260807 15:13:59.170125 12529 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.226.65:48021
W20260807 15:13:59.283527 12801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:59.283850 12801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:59.283901 12801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:59.288115 12801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:59.288197 12801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:59.288259 12801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.66
I20260807 15:13:59.293236 12801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.226.66
--webserver_port=0
--tserver_master_addrs=127.15.226.126:36843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.66
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:59.294412 12801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:59.295888 12801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:59.306910 12809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:59.307010 12801 server_base.cc:1034] running on GCE node
W20260807 15:13:59.306955 12806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:59.306943 12807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:59.307618 12801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:59.308327 12801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:59.309537 12801 hybrid_clock.cc:648] HybridClock initialized: now 1786115639309523 us; error 34 us; skew 500 ppm
I20260807 15:13:59.311719 12801 webserver.cc:460] Webserver started at http://127.15.226.66:44711/ using document root <none> and password file <none>
I20260807 15:13:59.312356 12801 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:59.312458 12801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:59.312711 12801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:13:59.315127 12801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-1/data/instance:
uuid: "6c9a35d696c94b3eab92e5db21ff6193"
format_stamp: "Formatted at 2026-08-07 15:13:59 on dist-test-slave-3kk6"
server_key: "bdf15a1d1a82eb0dad7c04c9890f73b1"
server_key_iv: "5ec2a2c1eb6c018af20b309640617ac9"
server_key_version: "encryptionkey@0"
I20260807 15:13:59.315769 12801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance:
uuid: "6c9a35d696c94b3eab92e5db21ff6193"
format_stamp: "Formatted at 2026-08-07 15:13:59 on dist-test-slave-3kk6"
server_key: "bdf15a1d1a82eb0dad7c04c9890f73b1"
server_key_iv: "5ec2a2c1eb6c018af20b309640617ac9"
server_key_version: "encryptionkey@0"
I20260807 15:13:59.320196 12801 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 15:13:59.324152 12817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:59.325682 12801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 15:13:59.325816 12801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-1/wal
uuid: "6c9a35d696c94b3eab92e5db21ff6193"
format_stamp: "Formatted at 2026-08-07 15:13:59 on dist-test-slave-3kk6"
server_key: "bdf15a1d1a82eb0dad7c04c9890f73b1"
server_key_iv: "5ec2a2c1eb6c018af20b309640617ac9"
server_key_version: "encryptionkey@0"
I20260807 15:13:59.325938 12801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:59.353981 12801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:59.354879 12801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:59.355073 12801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:59.355685 12801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:59.356698 12801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:59.356748 12801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:59.356801 12801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:59.356823 12801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:59.366485 12801 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.66:42791
I20260807 15:13:59.366513 12972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.66:42791 every 8 connection(s)
I20260807 15:13:59.367565 12801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-1/data/info.pb
I20260807 15:13:59.377743 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 12801
I20260807 15:13:59.377908 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-1/wal/instance
I20260807 15:13:59.378247 16265 external_mini_cluster.cc:1428] Setting key 97db703730a8c12787562ee3a325599b
I20260807 15:13:59.378268 12595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:59.369593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.66:57457 (local address 127.15.226.126:36843)
0807 15:13:59.369746 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:59.369749 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:59.371165 (+ 1416us) server_negotiation.cc:407] Connection header received
0807 15:13:59.372703 (+ 1538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:59.372706 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:59.372760 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:59.372846 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:59.373821 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.374378 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.376165 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.376426 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.376521 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:59.377709 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:59.377712 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:59.377715 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:13:59.377739 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:59.377761 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:13:59.377769 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:59.377860 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:59.378036 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 15:13:59.378116 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 15:13:59.379429 12975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:59.370038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36843 (local address 127.15.226.66:57457)
0807 15:13:59.370970 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:59.371030 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:59.372455 (+ 1425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:59.373029 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:59.373036 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:59.373124 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:59.373664 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:59.373678 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.374502 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.374504 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:59.376007 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:59.376020 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.376580 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.376583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:59.376742 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:59.377401 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:13:59.377403 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:59.377449 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:13:59.377526 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:59.377863 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:59.377876 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:59.377884 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 15:13:59.378036 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 15:13:59.378370 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":137,"threads_started":1}
I20260807 15:13:59.380640 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.226.67:0
--local_ip_for_outbound_sockets=127.15.226.67
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:36843
--builtin_ntp_servers=127.15.226.84:34429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 15:13:59.380797 12973 heartbeater.cc:347] Connected to a master server at 127.15.226.126:36843
I20260807 15:13:59.381069 12973 heartbeater.cc:464] Registering TS with master...
I20260807 15:13:59.381774 12973 heartbeater.cc:511] Master 127.15.226.126:36843 requested a full tablet report, sending...
I20260807 15:13:59.383124 12529 ts_manager.cc:194] Registered new tserver with Master: 6c9a35d696c94b3eab92e5db21ff6193 (127.15.226.66:42791)
I20260807 15:13:59.384021 12529 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.226.66:57457
W20260807 15:13:59.503306 12976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:13:59.503599 12976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:13:59.503643 12976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:13:59.508023 12976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:13:59.508105 12976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:13:59.508167 12976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.226.67
I20260807 15:13:59.513355 12976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:34429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.226.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.226.67
--webserver_port=0
--tserver_master_addrs=127.15.226.126:36843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.226.67
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:13:59.514572 12976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:13:59.516211 12976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:13:59.526794 12987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:59.527016 12976 server_base.cc:1034] running on GCE node
W20260807 15:13:59.526803 12985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:13:59.526806 12984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:13:59.527583 12976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:13:59.528352 12976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:13:59.529551 12976 hybrid_clock.cc:648] HybridClock initialized: now 1786115639529489 us; error 85 us; skew 500 ppm
I20260807 15:13:59.531894 12976 webserver.cc:460] Webserver started at http://127.15.226.67:40365/ using document root <none> and password file <none>
I20260807 15:13:59.532531 12976 fs_manager.cc:359] Metadata directory not provided
I20260807 15:13:59.532634 12976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:13:59.532889 12976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:13:59.535226 12976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-2/data/instance:
uuid: "4253dfc9a32346598db674429e66c345"
format_stamp: "Formatted at 2026-08-07 15:13:59 on dist-test-slave-3kk6"
server_key: "ad84ac97b6b15c906653a8ffa4bcd1ee"
server_key_iv: "17bb0caff771e04c34d1f8e7cd64ea69"
server_key_version: "encryptionkey@0"
I20260807 15:13:59.535889 12976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance:
uuid: "4253dfc9a32346598db674429e66c345"
format_stamp: "Formatted at 2026-08-07 15:13:59 on dist-test-slave-3kk6"
server_key: "ad84ac97b6b15c906653a8ffa4bcd1ee"
server_key_iv: "17bb0caff771e04c34d1f8e7cd64ea69"
server_key_version: "encryptionkey@0"
I20260807 15:13:59.540122 12976 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:13:59.543043 12996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:59.544437 12976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:13:59.544597 12976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-2/wal
uuid: "4253dfc9a32346598db674429e66c345"
format_stamp: "Formatted at 2026-08-07 15:13:59 on dist-test-slave-3kk6"
server_key: "ad84ac97b6b15c906653a8ffa4bcd1ee"
server_key_iv: "17bb0caff771e04c34d1f8e7cd64ea69"
server_key_version: "encryptionkey@0"
I20260807 15:13:59.544734 12976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:13:59.562386 12976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:13:59.563256 12976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:13:59.563499 12976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:13:59.564189 12976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 15:13:59.565238 12976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 15:13:59.565316 12976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:59.565402 12976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 15:13:59.565440 12976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 15:13:59.575480 12976 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.67:33893
I20260807 15:13:59.575526 13151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.67:33893 every 8 connection(s)
I20260807 15:13:59.576506 12976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-2/data/info.pb
I20260807 15:13:59.579854 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 12976
I20260807 15:13:59.579993 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-2/wal/instance
I20260807 15:13:59.580322 16265 external_mini_cluster.cc:1428] Setting key 87ae86bd9c9b76ba4c7982d58e96fbc4
I20260807 15:13:59.587217 12595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:59.578557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:41365 (local address 127.15.226.126:36843)
0807 15:13:59.578750 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:59.578754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:59.580140 (+ 1386us) server_negotiation.cc:407] Connection header received
0807 15:13:59.581983 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:59.581987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:59.582042 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:59.582135 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:59.583261 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.583802 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.585095 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.585313 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.585396 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:59.586745 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:59.586748 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:59.586750 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:13:59.586765 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:59.586788 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:13:59.586796 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:59.586861 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:59.587028 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 15:13:59.587094 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:13:59.588301 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:59.579046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:36843 (local address 127.15.226.67:41365)
0807 15:13:59.579934 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:59.580003 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:59.581737 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:59.582299 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:59.582306 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:59.582397 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:13:59.583119 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:59.583134 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.583951 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.583954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:59.584967 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:59.584978 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.585501 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.585505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:59.585684 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:59.586388 (+ 704us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 15:13:59.586390 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:13:59.586435 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 15:13:59.586512 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:13:59.586881 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:13:59.586889 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:13:59.586895 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 15:13:59.587010 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 15:13:59.587283 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":131,"threads_started":1}
I20260807 15:13:59.589715 13153 heartbeater.cc:347] Connected to a master server at 127.15.226.126:36843
I20260807 15:13:59.590018 13153 heartbeater.cc:464] Registering TS with master...
I20260807 15:13:59.590637 13153 heartbeater.cc:511] Master 127.15.226.126:36843 requested a full tablet report, sending...
I20260807 15:13:59.591930 12529 ts_manager.cc:194] Registered new tserver with Master: 4253dfc9a32346598db674429e66c345 (127.15.226.67:33893)
I20260807 15:13:59.592509 12529 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.226.67:41365
I20260807 15:13:59.594749 16265 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 15:13:59.600800 12595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:59.596799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45818 (local address 127.15.226.126:36843)
0807 15:13:59.596911 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:59.596914 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:59.597165 (+ 251us) server_negotiation.cc:407] Connection header received
0807 15:13:59.597261 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:59.597263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:59.597307 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:59.597387 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:13:59.597605 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.598114 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.598995 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.599206 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.599285 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:59.600332 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:13:59.600336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:13:59.600337 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:13:59.600351 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:13:59.600372 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:13:59.600381 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:13:59.600449 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:59.600549 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 15:13:59.600604 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 15:13:59.604264 12529 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45818:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 15:13:59.607609 12529 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 15:13:59.623775 13172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:59.617347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33228 (local address 127.15.226.67:33893)
0807 15:13:59.618500 (+ 1153us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:59.618513 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:59.618538 (+ 25us) server_negotiation.cc:407] Connection header received
0807 15:13:59.618643 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:59.618681 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:59.618901 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:59.619051 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:59.619669 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.620558 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.622085 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.622979 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.623098 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:59.623330 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:59.623457 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 15:13:59.623572 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1043,"thread_start_us":194,"threads_started":1}
I20260807 15:13:59.623777 13176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:59.617183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.67:33893 (local address 127.0.0.1:33228)
0807 15:13:59.618095 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:59.618157 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:59.618337 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:59.619239 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:59.619245 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:59.619283 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:59.619524 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:59.619529 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.620683 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.620688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:59.621934 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:59.621945 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.623120 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.623125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:59.623253 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:59.623303 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 15:13:59.623420 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 15:13:59.623585 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":144,"spinlock_wait_cycles":4736,"thread_start_us":134,"threads_started":1}
I20260807 15:13:59.623778 13171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:59.617990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:44729 (local address 127.0.0.1:51122)
0807 15:13:59.618757 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:59.618771 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:59.618850 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:59.619539 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:59.619543 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:59.619561 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:59.619681 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:59.619702 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.621184 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.621188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:59.622279 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:59.622304 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.623272 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.623275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:59.623345 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:59.623367 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 15:13:59.623415 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 15:13:59.623585 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"mutex_wait_us":3,"thread_start_us":120,"threads_started":1}
I20260807 15:13:59.624361 13178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:59.617709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33526 (local address 127.15.226.66:42791)
0807 15:13:59.618064 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:59.618077 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:59.618405 (+ 328us) server_negotiation.cc:407] Connection header received
0807 15:13:59.618706 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:59.618711 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:59.618901 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:59.619042 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:59.619948 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.621097 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.622530 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.623362 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.623473 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:59.623652 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:59.624068 (+ 416us) server_negotiation.cc:299] Negotiation successful
0807 15:13:59.624172 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":137,"threads_started":1}
I20260807 15:13:59.624466 13180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:59.618455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51122 (local address 127.15.226.65:44729)
0807 15:13:59.619108 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:59.619114 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:59.619139 (+ 25us) server_negotiation.cc:407] Connection header received
0807 15:13:59.619231 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:59.619236 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:59.619392 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:59.619561 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:59.619832 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.621030 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.622435 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.623118 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.623861 (+ 743us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:59.624068 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:59.624127 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 15:13:59.624257 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":100,"threads_started":1}
I20260807 15:13:59.625005 13179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:59.617474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:42791 (local address 127.0.0.1:33526)
0807 15:13:59.618269 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:59.618285 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:59.618490 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:59.619239 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:59.619245 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:59.619284 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:59.619519 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:59.619527 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.621280 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.621283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:59.622306 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:59.622324 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.623727 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.623730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:59.623865 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:59.623901 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 15:13:59.624717 (+ 816us) client_negotiation.cc:240] Negotiation successful
0807 15:13:59.624834 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"mutex_wait_us":21,"spinlock_wait_cycles":7168,"thread_start_us":99,"threads_started":1}
I20260807 15:13:59.627894 12703 tablet_service.cc:1467] Processing CreateTablet for tablet 61d1be455b954f1f9c2dc9fa70317ee3 (DEFAULT_TABLE table=test-table [id=17df2e8fbd98404f8c536e2b836b25b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:13:59.627960 13063 tablet_service.cc:1467] Processing CreateTablet for tablet 61d1be455b954f1f9c2dc9fa70317ee3 (DEFAULT_TABLE table=test-table [id=17df2e8fbd98404f8c536e2b836b25b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:13:59.627960 12888 tablet_service.cc:1467] Processing CreateTablet for tablet 61d1be455b954f1f9c2dc9fa70317ee3 (DEFAULT_TABLE table=test-table [id=17df2e8fbd98404f8c536e2b836b25b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 15:13:59.629060 12703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61d1be455b954f1f9c2dc9fa70317ee3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:59.629065 12888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61d1be455b954f1f9c2dc9fa70317ee3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:59.629065 13063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61d1be455b954f1f9c2dc9fa70317ee3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:13:59.636163 13181 tablet_bootstrap.cc:492] T 61d1be455b954f1f9c2dc9fa70317ee3 P f8b539e910be40779693c54e7da371c2: Bootstrap starting.
I20260807 15:13:59.637907 13182 tablet_bootstrap.cc:492] T 61d1be455b954f1f9c2dc9fa70317ee3 P 6c9a35d696c94b3eab92e5db21ff6193: Bootstrap starting.
I20260807 15:13:59.638585 13181 tablet_bootstrap.cc:654] T 61d1be455b954f1f9c2dc9fa70317ee3 P f8b539e910be40779693c54e7da371c2: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:59.639039 13183 tablet_bootstrap.cc:492] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345: Bootstrap starting.
I20260807 15:13:59.639484 13181 log.cc:824] T 61d1be455b954f1f9c2dc9fa70317ee3 P f8b539e910be40779693c54e7da371c2: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:59.640036 13182 tablet_bootstrap.cc:654] T 61d1be455b954f1f9c2dc9fa70317ee3 P 6c9a35d696c94b3eab92e5db21ff6193: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:59.640882 13182 log.cc:824] T 61d1be455b954f1f9c2dc9fa70317ee3 P 6c9a35d696c94b3eab92e5db21ff6193: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:59.641250 13183 tablet_bootstrap.cc:654] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345: Neither blocks nor log segments found. Creating new log.
I20260807 15:13:59.641986 13183 log.cc:824] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345: Log is configured to *not* fsync() on all Append() calls
I20260807 15:13:59.642194 13181 tablet_bootstrap.cc:492] T 61d1be455b954f1f9c2dc9fa70317ee3 P f8b539e910be40779693c54e7da371c2: No bootstrap required, opened a new log
I20260807 15:13:59.642441 13181 ts_tablet_manager.cc:1397] T 61d1be455b954f1f9c2dc9fa70317ee3 P f8b539e910be40779693c54e7da371c2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 15:13:59.643188 13182 tablet_bootstrap.cc:492] T 61d1be455b954f1f9c2dc9fa70317ee3 P 6c9a35d696c94b3eab92e5db21ff6193: No bootstrap required, opened a new log
I20260807 15:13:59.643390 13182 ts_tablet_manager.cc:1397] T 61d1be455b954f1f9c2dc9fa70317ee3 P 6c9a35d696c94b3eab92e5db21ff6193: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 15:13:59.644052 13183 tablet_bootstrap.cc:492] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345: No bootstrap required, opened a new log
I20260807 15:13:59.644248 13183 ts_tablet_manager.cc:1397] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 15:13:59.646450 13181 raft_consensus.cc:348] T 61d1be455b954f1f9c2dc9fa70317ee3 P f8b539e910be40779693c54e7da371c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4253dfc9a32346598db674429e66c345" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 33893 } } peers { permanent_uuid: "f8b539e910be40779693c54e7da371c2" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44729 } } peers { permanent_uuid: "6c9a35d696c94b3eab92e5db21ff6193" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42791 } }
I20260807 15:13:59.646816 13181 raft_consensus.cc:374] T 61d1be455b954f1f9c2dc9fa70317ee3 P f8b539e910be40779693c54e7da371c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:59.646878 13181 raft_consensus.cc:729] T 61d1be455b954f1f9c2dc9fa70317ee3 P f8b539e910be40779693c54e7da371c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8b539e910be40779693c54e7da371c2, State: Initialized, Role: FOLLOWER
I20260807 15:13:59.647384 13181 consensus_queue.cc:260] T 61d1be455b954f1f9c2dc9fa70317ee3 P f8b539e910be40779693c54e7da371c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4253dfc9a32346598db674429e66c345" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 33893 } } peers { permanent_uuid: "f8b539e910be40779693c54e7da371c2" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44729 } } peers { permanent_uuid: "6c9a35d696c94b3eab92e5db21ff6193" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42791 } }
I20260807 15:13:59.647835 13182 raft_consensus.cc:348] T 61d1be455b954f1f9c2dc9fa70317ee3 P 6c9a35d696c94b3eab92e5db21ff6193 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4253dfc9a32346598db674429e66c345" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 33893 } } peers { permanent_uuid: "f8b539e910be40779693c54e7da371c2" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44729 } } peers { permanent_uuid: "6c9a35d696c94b3eab92e5db21ff6193" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42791 } }
I20260807 15:13:59.648097 13182 raft_consensus.cc:374] T 61d1be455b954f1f9c2dc9fa70317ee3 P 6c9a35d696c94b3eab92e5db21ff6193 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:59.648116 12798 heartbeater.cc:503] Master 127.15.226.126:36843 was elected leader, sending a full tablet report...
I20260807 15:13:59.648160 13182 raft_consensus.cc:729] T 61d1be455b954f1f9c2dc9fa70317ee3 P 6c9a35d696c94b3eab92e5db21ff6193 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c9a35d696c94b3eab92e5db21ff6193, State: Initialized, Role: FOLLOWER
I20260807 15:13:59.648566 13181 ts_tablet_manager.cc:1428] T 61d1be455b954f1f9c2dc9fa70317ee3 P f8b539e910be40779693c54e7da371c2: Time spent starting tablet: real 0.006s user 0.004s sys 0.004s
I20260807 15:13:59.648676 13182 consensus_queue.cc:260] T 61d1be455b954f1f9c2dc9fa70317ee3 P 6c9a35d696c94b3eab92e5db21ff6193 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4253dfc9a32346598db674429e66c345" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 33893 } } peers { permanent_uuid: "f8b539e910be40779693c54e7da371c2" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44729 } } peers { permanent_uuid: "6c9a35d696c94b3eab92e5db21ff6193" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42791 } }
I20260807 15:13:59.648636 13183 raft_consensus.cc:348] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4253dfc9a32346598db674429e66c345" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 33893 } } peers { permanent_uuid: "f8b539e910be40779693c54e7da371c2" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44729 } } peers { permanent_uuid: "6c9a35d696c94b3eab92e5db21ff6193" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42791 } }
I20260807 15:13:59.649088 13183 raft_consensus.cc:374] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:13:59.649152 13183 raft_consensus.cc:729] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4253dfc9a32346598db674429e66c345, State: Initialized, Role: FOLLOWER
I20260807 15:13:59.649534 12973 heartbeater.cc:503] Master 127.15.226.126:36843 was elected leader, sending a full tablet report...
I20260807 15:13:59.649714 13182 ts_tablet_manager.cc:1428] T 61d1be455b954f1f9c2dc9fa70317ee3 P 6c9a35d696c94b3eab92e5db21ff6193: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 15:13:59.649657 13183 consensus_queue.cc:260] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4253dfc9a32346598db674429e66c345" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 33893 } } peers { permanent_uuid: "f8b539e910be40779693c54e7da371c2" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44729 } } peers { permanent_uuid: "6c9a35d696c94b3eab92e5db21ff6193" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42791 } }
I20260807 15:13:59.650336 13153 heartbeater.cc:503] Master 127.15.226.126:36843 was elected leader, sending a full tablet report...
I20260807 15:13:59.650614 13183 ts_tablet_manager.cc:1428] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260807 15:13:59.652032 12799 tablet.cc:2369] T 61d1be455b954f1f9c2dc9fa70317ee3 P f8b539e910be40779693c54e7da371c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:13:59.827730 13154 tablet.cc:2369] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 15:13:59.868830 12974 tablet.cc:2369] T 61d1be455b954f1f9c2dc9fa70317ee3 P 6c9a35d696c94b3eab92e5db21ff6193: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 15:13:59.875590 13193 raft_consensus.cc:482] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 15:13:59.875900 13193 raft_consensus.cc:504] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4253dfc9a32346598db674429e66c345" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 33893 } } peers { permanent_uuid: "f8b539e910be40779693c54e7da371c2" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44729 } } peers { permanent_uuid: "6c9a35d696c94b3eab92e5db21ff6193" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42791 } }
I20260807 15:13:59.877174 13193 leader_election.cc:288] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8b539e910be40779693c54e7da371c2 (127.15.226.65:44729), 6c9a35d696c94b3eab92e5db21ff6193 (127.15.226.66:42791)
I20260807 15:13:59.882001 13158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:59.877449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:44729 (local address 127.15.226.67:40035)
0807 15:13:59.877668 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:59.877687 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:59.877829 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:59.878246 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:59.878249 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:59.878262 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:59.878404 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:59.878413 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.879499 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.879502 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:59.880760 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:59.880772 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.881487 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.881491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:59.881606 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:59.881656 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 15:13:59.881733 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 15:13:59.881813 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":35}
I20260807 15:13:59.882025 13180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:59.877594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:40035 (local address 127.15.226.65:44729)
0807 15:13:59.877751 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:59.877756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:59.877777 (+ 21us) server_negotiation.cc:407] Connection header received
0807 15:13:59.877988 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:59.877994 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:59.878054 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:59.878128 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:59.878546 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.879369 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.880945 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.881354 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.881489 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:59.881542 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:59.881804 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 15:13:59.881894 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 15:13:59.882172 13178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:59.878011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.226.67:52917 (local address 127.15.226.66:42791)
0807 15:13:59.878143 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:59.878147 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:59.878159 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:13:59.878213 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:59.878216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:59.878265 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:59.878354 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 15:13:59.878720 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.879215 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.880929 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.881564 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.881681 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:59.881729 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:59.881961 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 15:13:59.882052 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 15:13:59.882550 13194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:59.877507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.66:42791 (local address 127.15.226.67:52917)
0807 15:13:59.877942 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0807 15:13:59.877959 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:13:59.878036 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:13:59.878388 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:13:59.878391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:13:59.878404 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 15:13:59.878556 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:59.878562 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.879344 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.879348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:59.880760 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:13:59.880772 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.881705 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.881712 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:13:59.881803 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:59.881823 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 15:13:59.882305 (+ 482us) client_negotiation.cc:240] Negotiation successful
0807 15:13:59.882384 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":32,"thread_start_us":172,"threads_started":1}
I20260807 15:13:59.882809 12732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61d1be455b954f1f9c2dc9fa70317ee3" candidate_uuid: "4253dfc9a32346598db674429e66c345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8b539e910be40779693c54e7da371c2" is_pre_election: true
I20260807 15:13:59.883201 12732 raft_consensus.cc:2454] T 61d1be455b954f1f9c2dc9fa70317ee3 P f8b539e910be40779693c54e7da371c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4253dfc9a32346598db674429e66c345 in term 0.
I20260807 15:13:59.883283 12912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61d1be455b954f1f9c2dc9fa70317ee3" candidate_uuid: "4253dfc9a32346598db674429e66c345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c9a35d696c94b3eab92e5db21ff6193" is_pre_election: true
I20260807 15:13:59.883599 12912 raft_consensus.cc:2454] T 61d1be455b954f1f9c2dc9fa70317ee3 P 6c9a35d696c94b3eab92e5db21ff6193 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4253dfc9a32346598db674429e66c345 in term 0.
I20260807 15:13:59.883788 12999 leader_election.cc:302] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4253dfc9a32346598db674429e66c345, f8b539e910be40779693c54e7da371c2; no voters:
I20260807 15:13:59.884116 13193 raft_consensus.cc:2781] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 15:13:59.884210 13193 raft_consensus.cc:482] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 15:13:59.884263 13193 raft_consensus.cc:3037] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:59.885385 13193 raft_consensus.cc:504] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4253dfc9a32346598db674429e66c345" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 33893 } } peers { permanent_uuid: "f8b539e910be40779693c54e7da371c2" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44729 } } peers { permanent_uuid: "6c9a35d696c94b3eab92e5db21ff6193" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42791 } }
I20260807 15:13:59.885821 13193 leader_election.cc:288] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345 [CANDIDATE]: Term 1 election: Requested vote from peers f8b539e910be40779693c54e7da371c2 (127.15.226.65:44729), 6c9a35d696c94b3eab92e5db21ff6193 (127.15.226.66:42791)
I20260807 15:13:59.886171 12732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61d1be455b954f1f9c2dc9fa70317ee3" candidate_uuid: "4253dfc9a32346598db674429e66c345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8b539e910be40779693c54e7da371c2"
I20260807 15:13:59.886204 12912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61d1be455b954f1f9c2dc9fa70317ee3" candidate_uuid: "4253dfc9a32346598db674429e66c345" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c9a35d696c94b3eab92e5db21ff6193"
I20260807 15:13:59.886307 12732 raft_consensus.cc:3037] T 61d1be455b954f1f9c2dc9fa70317ee3 P f8b539e910be40779693c54e7da371c2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:59.886356 12912 raft_consensus.cc:3037] T 61d1be455b954f1f9c2dc9fa70317ee3 P 6c9a35d696c94b3eab92e5db21ff6193 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:13:59.887547 12912 raft_consensus.cc:2454] T 61d1be455b954f1f9c2dc9fa70317ee3 P 6c9a35d696c94b3eab92e5db21ff6193 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4253dfc9a32346598db674429e66c345 in term 1.
I20260807 15:13:59.887740 12732 raft_consensus.cc:2454] T 61d1be455b954f1f9c2dc9fa70317ee3 P f8b539e910be40779693c54e7da371c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4253dfc9a32346598db674429e66c345 in term 1.
I20260807 15:13:59.887929 13005 leader_election.cc:302] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4253dfc9a32346598db674429e66c345, 6c9a35d696c94b3eab92e5db21ff6193; no voters:
I20260807 15:13:59.888157 13193 raft_consensus.cc:2781] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:13:59.888417 13193 raft_consensus.cc:686] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345 [term 1 LEADER]: Becoming Leader. State: Replica: 4253dfc9a32346598db674429e66c345, State: Running, Role: LEADER
I20260807 15:13:59.888808 13193 consensus_queue.cc:237] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4253dfc9a32346598db674429e66c345" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 33893 } } peers { permanent_uuid: "f8b539e910be40779693c54e7da371c2" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44729 } } peers { permanent_uuid: "6c9a35d696c94b3eab92e5db21ff6193" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42791 } }
I20260807 15:13:59.892673 12529 catalog_manager.cc:5495] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4253dfc9a32346598db674429e66c345 (127.15.226.67). New cstate: current_term: 1 leader_uuid: "4253dfc9a32346598db674429e66c345" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4253dfc9a32346598db674429e66c345" member_type: VOTER last_known_addr { host: "127.15.226.67" port: 33893 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f8b539e910be40779693c54e7da371c2" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c9a35d696c94b3eab92e5db21ff6193" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42791 } health_report { overall_health: UNKNOWN } } }
I20260807 15:13:59.926719 13172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:13:59.923000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33236 (local address 127.15.226.67:33893)
0807 15:13:59.923183 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 15:13:59.923187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:13:59.923202 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:13:59.923310 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:13:59.923312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:13:59.923383 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:13:59.923478 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:13:59.923713 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.924215 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.925086 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:13:59.925299 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:13:59.925387 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:13:59.925688 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:13:59.926146 (+ 458us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:13:59.926241 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 15:13:59.926406 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 15:13:59.926499 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 15:13:59.933614 12732 raft_consensus.cc:1261] T 61d1be455b954f1f9c2dc9fa70317ee3 P f8b539e910be40779693c54e7da371c2 [term 1 FOLLOWER]: Refusing update from remote peer 4253dfc9a32346598db674429e66c345: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:13:59.933614 12912 raft_consensus.cc:1261] T 61d1be455b954f1f9c2dc9fa70317ee3 P 6c9a35d696c94b3eab92e5db21ff6193 [term 1 FOLLOWER]: Refusing update from remote peer 4253dfc9a32346598db674429e66c345: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:13:59.934419 13196 consensus_queue.cc:1035] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8b539e910be40779693c54e7da371c2" member_type: VOTER last_known_addr { host: "127.15.226.65" port: 44729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:13:59.934638 13193 consensus_queue.cc:1035] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c9a35d696c94b3eab92e5db21ff6193" member_type: VOTER last_known_addr { host: "127.15.226.66" port: 42791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:13:59.942094 13199 mvcc.cc:205] Tried to move back new op lower bound from 7315929661159751680 to 7315929660987449344. Current Snapshot: MvccSnapshot[applied={T|T < 7315929661159751680}]
I20260807 15:13:59.943285 13202 mvcc.cc:205] Tried to move back new op lower bound from 7315929661159751680 to 7315929660987449344. Current Snapshot: MvccSnapshot[applied={T|T < 7315929661159751680}]
I20260807 15:13:59.955264 12529 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:13:59.955502 12529 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45818:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 15:13:59.958211 12529 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86ace2792b2f44ae99bcf9ec4f652641: Sending DeleteTablet for 3 replicas of tablet 61d1be455b954f1f9c2dc9fa70317ee3
I20260807 15:13:59.959247 13063 tablet_service.cc:1514] Processing DeleteTablet for tablet 61d1be455b954f1f9c2dc9fa70317ee3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:13:59 UTC) from {username='slave'} at 127.0.0.1:33228
I20260807 15:13:59.959282 12703 tablet_service.cc:1514] Processing DeleteTablet for tablet 61d1be455b954f1f9c2dc9fa70317ee3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:13:59 UTC) from {username='slave'} at 127.0.0.1:51122
I20260807 15:13:59.959848 13211 tablet_replica.cc:330] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345: stopping tablet replica
I20260807 15:13:59.959884 12888 tablet_service.cc:1514] Processing DeleteTablet for tablet 61d1be455b954f1f9c2dc9fa70317ee3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 15:13:59 UTC) from {username='slave'} at 127.0.0.1:33526
I20260807 15:13:59.960227 13210 tablet_replica.cc:330] T 61d1be455b954f1f9c2dc9fa70317ee3 P f8b539e910be40779693c54e7da371c2: stopping tablet replica
I20260807 15:13:59.960249 13211 raft_consensus.cc:2229] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345 [term 1 LEADER]: Raft consensus shutting down.
I20260807 15:13:59.960646 13210 raft_consensus.cc:2229] T 61d1be455b954f1f9c2dc9fa70317ee3 P f8b539e910be40779693c54e7da371c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:13:59.960759 13211 raft_consensus.cc:2258] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:13:59.961139 13210 raft_consensus.cc:2258] T 61d1be455b954f1f9c2dc9fa70317ee3 P f8b539e910be40779693c54e7da371c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:13:59.961273 13212 tablet_replica.cc:330] T 61d1be455b954f1f9c2dc9fa70317ee3 P 6c9a35d696c94b3eab92e5db21ff6193: stopping tablet replica
I20260807 15:13:59.961594 13212 raft_consensus.cc:2229] T 61d1be455b954f1f9c2dc9fa70317ee3 P 6c9a35d696c94b3eab92e5db21ff6193 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 15:13:59.961630 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 12621
I20260807 15:13:59.961923 13212 raft_consensus.cc:2258] T 61d1be455b954f1f9c2dc9fa70317ee3 P 6c9a35d696c94b3eab92e5db21ff6193 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 15:13:59.968752 13212 ts_tablet_manager.cc:1905] T 61d1be455b954f1f9c2dc9fa70317ee3 P 6c9a35d696c94b3eab92e5db21ff6193: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:13:59.972328 13212 ts_tablet_manager.cc:1918] T 61d1be455b954f1f9c2dc9fa70317ee3 P 6c9a35d696c94b3eab92e5db21ff6193: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:13:59.972474 13212 log.cc:1196] T 61d1be455b954f1f9c2dc9fa70317ee3 P 6c9a35d696c94b3eab92e5db21ff6193: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-1/wal/wals/61d1be455b954f1f9c2dc9fa70317ee3
I20260807 15:13:59.972856 13212 ts_tablet_manager.cc:1939] T 61d1be455b954f1f9c2dc9fa70317ee3 P 6c9a35d696c94b3eab92e5db21ff6193: Deleting consensus metadata
I20260807 15:13:59.974416 12509 catalog_manager.cc:4841] TS 6c9a35d696c94b3eab92e5db21ff6193 (127.15.226.66:42791): tablet 61d1be455b954f1f9c2dc9fa70317ee3 (table test-table [id=17df2e8fbd98404f8c536e2b836b25b7]) successfully deleted
I20260807 15:13:59.978729 13211 ts_tablet_manager.cc:1905] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 15:13:59.981469 13211 ts_tablet_manager.cc:1918] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 15:13:59.981631 13211 log.cc:1196] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345: Deleting WAL directory at /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786115365131938-16265-0/minicluster-data/ts-2/wal/wals/61d1be455b954f1f9c2dc9fa70317ee3
I20260807 15:13:59.981973 13211 ts_tablet_manager.cc:1939] T 61d1be455b954f1f9c2dc9fa70317ee3 P 4253dfc9a32346598db674429e66c345: Deleting consensus metadata
I20260807 15:13:59.983439 12507 catalog_manager.cc:4841] TS 4253dfc9a32346598db674429e66c345 (127.15.226.67:33893): tablet 61d1be455b954f1f9c2dc9fa70317ee3 (table test-table [id=17df2e8fbd98404f8c536e2b836b25b7]) successfully deleted
I20260807 15:13:59.987720 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 12801
W20260807 15:13:59.987756 12507 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.226.65:44729 (error 108)
I20260807 15:14:00.000849 13179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:00.000323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.65:44729 (local address 127.0.0.1:51134)
0807 15:14:00.000607 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:00.000717 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.65:44729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
W20260807 15:14:00.001185 12507 catalog_manager.cc:4584] TS f8b539e910be40779693c54e7da371c2 (127.15.226.65:44729): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 61d1be455b954f1f9c2dc9fa70317ee3: Network error: Client connection negotiation failed: client connection to 127.15.226.65:44729: connect: Connection refused (error 111)
I20260807 15:14:00.006422 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 12976
I20260807 15:14:00.023290 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 12489
2026-08-07T15:14:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1392 ms)
[----------] 6 tests from EncryptionPolicyTest (12316 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:00 dist-test-slave-3kk6 krb5kdc[13217](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:00 dist-test-slave-3kk6 krb5kdc[13217](info): set up 2 sockets
Aug 07 15:14:00 dist-test-slave-3kk6 krb5kdc[13217](info): commencing operation
krb5kdc: starting...
W20260807 15:14:01.273073 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:14:01 dist-test-slave-3kk6 krb5kdc[13217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:14:01.414244 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46737
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:01.534320 13231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:01.534735 13231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:01.534809 13231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:01.539021 13231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:01.539161 13231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:01.539201 13231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:01.539242 13231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:01.539320 13231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:01.545730 13231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46737
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:01.547420 13231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:01.548702 13231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:01.558561 13238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:01.558573 13237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:01.558565 13240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:01.558908 13231 server_base.cc:1034] running on GCE node
I20260807 15:14:01.559705 13231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:01.560964 13231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:01.562142 13231 hybrid_clock.cc:648] HybridClock initialized: now 1786115641562113 us; error 49 us; skew 500 ppm
Aug 07 15:14:01 dist-test-slave-3kk6 krb5kdc[13217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115641, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:01.571036 13231 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:14:01.572260 13231 webserver.cc:460] Webserver started at http://127.0.0.1:36929/ using document root <none> and password file <none>
I20260807 15:14:01.572921 13231 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:01.572994 13231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:01.573262 13231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:01.575699 13231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "f52233af8dbe48429a7f8e8e0a2f01fe"
format_stamp: "Formatted at 2026-08-07 15:14:01 on dist-test-slave-3kk6"
server_key: "395fe52241b00326925b9f96929621c0"
server_key_iv: "f54be4da74ba529ff3a20683cf4924b0"
server_key_version: "encryptionkey@0"
I20260807 15:14:01.576370 13231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "f52233af8dbe48429a7f8e8e0a2f01fe"
format_stamp: "Formatted at 2026-08-07 15:14:01 on dist-test-slave-3kk6"
server_key: "395fe52241b00326925b9f96929621c0"
server_key_iv: "f54be4da74ba529ff3a20683cf4924b0"
server_key_version: "encryptionkey@0"
I20260807 15:14:01.580724 13231 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:14:01.583516 13248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:01.584972 13231 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:14:01.585180 13231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "f52233af8dbe48429a7f8e8e0a2f01fe"
format_stamp: "Formatted at 2026-08-07 15:14:01 on dist-test-slave-3kk6"
server_key: "395fe52241b00326925b9f96929621c0"
server_key_iv: "f54be4da74ba529ff3a20683cf4924b0"
server_key_version: "encryptionkey@0"
I20260807 15:14:01.585343 13231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:01.597282 13231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:01.600741 13231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:01.601001 13231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:01.609689 13231 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46737
I20260807 15:14:01.609731 13318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46737 every 8 connection(s)
I20260807 15:14:01.610904 13231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:01.613101 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 13231
I20260807 15:14:01.613219 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:14:01.613586 16265 external_mini_cluster.cc:1428] Setting key 1375cf086b9a290cb871b5bcb8bc0bea
I20260807 15:14:01.614125 13321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:01.620922 13321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f52233af8dbe48429a7f8e8e0a2f01fe: Bootstrap starting.
I20260807 15:14:01.623732 13321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f52233af8dbe48429a7f8e8e0a2f01fe: Neither blocks nor log segments found. Creating new log.
Aug 07 15:14:01 dist-test-slave-3kk6 krb5kdc[13217](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 15:14:01.624620 13321 log.cc:824] T 00000000000000000000000000000000 P f52233af8dbe48429a7f8e8e0a2f01fe: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:01.627065 13321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f52233af8dbe48429a7f8e8e0a2f01fe: No bootstrap required, opened a new log
I20260807 15:14:01.629824 13324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:01.615244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39809 (local address 127.0.0.1:46737)
0807 15:14:01.615814 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:01.615832 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:01.615874 (+ 42us) server_negotiation.cc:407] Connection header received
0807 15:14:01.616602 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:01.616621 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:01.617000 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:01.617411 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:01.618344 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:01.619610 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:01.620631 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:01.621255 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:01.621447 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:01.624665 (+ 3218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:01.624670 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:01.624685 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:01.624749 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:01.626627 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:01.627279 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:01.627292 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:01.627298 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:01.627415 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:01.627829 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:01.627833 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:01.627835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:01.628260 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:01.628505 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:01.628824 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 15:14:01.629064 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":166,"threads_started":1}
I20260807 15:14:01.631315 13321 raft_consensus.cc:348] T 00000000000000000000000000000000 P f52233af8dbe48429a7f8e8e0a2f01fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52233af8dbe48429a7f8e8e0a2f01fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46737 } }
I20260807 15:14:01.631590 13321 raft_consensus.cc:374] T 00000000000000000000000000000000 P f52233af8dbe48429a7f8e8e0a2f01fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:01.631676 13321 raft_consensus.cc:729] T 00000000000000000000000000000000 P f52233af8dbe48429a7f8e8e0a2f01fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f52233af8dbe48429a7f8e8e0a2f01fe, State: Initialized, Role: FOLLOWER
I20260807 15:14:01.632256 13321 consensus_queue.cc:260] T 00000000000000000000000000000000 P f52233af8dbe48429a7f8e8e0a2f01fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52233af8dbe48429a7f8e8e0a2f01fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46737 } }
I20260807 15:14:01.632434 13321 raft_consensus.cc:388] T 00000000000000000000000000000000 P f52233af8dbe48429a7f8e8e0a2f01fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:01.632529 13321 raft_consensus.cc:482] T 00000000000000000000000000000000 P f52233af8dbe48429a7f8e8e0a2f01fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:01.632640 13321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f52233af8dbe48429a7f8e8e0a2f01fe [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:01.633282 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:14:01.633895 13321 raft_consensus.cc:504] T 00000000000000000000000000000000 P f52233af8dbe48429a7f8e8e0a2f01fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52233af8dbe48429a7f8e8e0a2f01fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46737 } }
I20260807 15:14:01.634279 13321 leader_election.cc:302] T 00000000000000000000000000000000 P f52233af8dbe48429a7f8e8e0a2f01fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f52233af8dbe48429a7f8e8e0a2f01fe; no voters:
I20260807 15:14:01.634641 13321 leader_election.cc:288] T 00000000000000000000000000000000 P f52233af8dbe48429a7f8e8e0a2f01fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:01.634840 13330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f52233af8dbe48429a7f8e8e0a2f01fe [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:01.635051 13330 raft_consensus.cc:686] T 00000000000000000000000000000000 P f52233af8dbe48429a7f8e8e0a2f01fe [term 1 LEADER]: Becoming Leader. State: Replica: f52233af8dbe48429a7f8e8e0a2f01fe, State: Running, Role: LEADER
I20260807 15:14:01.635371 13330 consensus_queue.cc:237] T 00000000000000000000000000000000 P f52233af8dbe48429a7f8e8e0a2f01fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52233af8dbe48429a7f8e8e0a2f01fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46737 } }
I20260807 15:14:01.636111 13321 sys_catalog.cc:564] T 00000000000000000000000000000000 P f52233af8dbe48429a7f8e8e0a2f01fe [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:01.636662 13330 sys_catalog.cc:455] T 00000000000000000000000000000000 P f52233af8dbe48429a7f8e8e0a2f01fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader f52233af8dbe48429a7f8e8e0a2f01fe. Latest consensus state: current_term: 1 leader_uuid: "f52233af8dbe48429a7f8e8e0a2f01fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52233af8dbe48429a7f8e8e0a2f01fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46737 } } }
I20260807 15:14:01.636791 13330 sys_catalog.cc:458] T 00000000000000000000000000000000 P f52233af8dbe48429a7f8e8e0a2f01fe [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:01.637230 13340 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:01.637334 13334 sys_catalog.cc:455] T 00000000000000000000000000000000 P f52233af8dbe48429a7f8e8e0a2f01fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f52233af8dbe48429a7f8e8e0a2f01fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f52233af8dbe48429a7f8e8e0a2f01fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46737 } } }
I20260807 15:14:01.637516 13334 sys_catalog.cc:458] T 00000000000000000000000000000000 P f52233af8dbe48429a7f8e8e0a2f01fe [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:01.640910 13340 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:01.646975 13324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:01.635666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44311 (local address 127.0.0.1:46737)
0807 15:14:01.635790 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:01.635795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:01.636138 (+ 343us) server_negotiation.cc:407] Connection header received
0807 15:14:01.636291 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:01.636294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:01.636348 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:01.636434 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:01.637385 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:01.638212 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:01.639266 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:01.639640 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:01.639752 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:01.642721 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:01.642727 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:01.642731 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:01.642762 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:01.644624 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:01.645349 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:01.645354 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:01.645358 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:01.645474 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:01.645906 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:01.645909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:01.645911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:01.646120 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:01.646300 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:01.646702 (+ 402us) server_negotiation.cc:299] Negotiation successful
0807 15:14:01.646815 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 15:14:01.648155 13340 catalog_manager.cc:1347] Generated new cluster ID: 04397a9a3f0c4f78a231df024f2ce9fb
I20260807 15:14:01.648238 13340 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:01.667266 13340 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:01.668342 13340 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:01.676383 13340 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f52233af8dbe48429a7f8e8e0a2f01fe: Generated new TSK 0
I20260807 15:14:01.677266 13340 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:14:01.681733 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 13231
W20260807 15:14:01.696647 13215 connection.cc:535] client connection to 127.0.0.1:46737 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46737): Connection reset by peer (error 104)
2026-08-07T15:14:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1659 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:01 dist-test-slave-3kk6 krb5kdc[13362](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:01 dist-test-slave-3kk6 krb5kdc[13362](info): set up 2 sockets
Aug 07 15:14:01 dist-test-slave-3kk6 krb5kdc[13362](info): commencing operation
krb5kdc: starting...
W20260807 15:14:02.933833 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:14:03 dist-test-slave-3kk6 krb5kdc[13362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:14:03.078454 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32795
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:03.201761 13379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:03.202147 13379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:03.202248 13379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:03.206583 13379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:03.206739 13379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:03.206791 13379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:03.206833 13379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:03.206885 13379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:03.212730 13379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32795
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:03.214012 13379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:03.215188 13379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:14:03.224524 13379 server_base.cc:1034] running on GCE node
W20260807 15:14:03.224510 13385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:03.224531 13390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:03.224498 13386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:03.225453 13379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:03.226756 13379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:03.227990 13379 hybrid_clock.cc:648] HybridClock initialized: now 1786115643227979 us; error 36 us; skew 500 ppm
Aug 07 15:14:03 dist-test-slave-3kk6 krb5kdc[13362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115643, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:03.231230 13379 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:14:03.232304 13379 webserver.cc:460] Webserver started at http://127.0.0.1:37733/ using document root <none> and password file <none>
I20260807 15:14:03.232932 13379 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:03.233033 13379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:03.233284 13379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:03.235687 13379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "80c9ad1daf924b26bde7cae22107964e"
format_stamp: "Formatted at 2026-08-07 15:14:03 on dist-test-slave-3kk6"
server_key: "f3dc4d3c0c0d2a6ef40788c3a92a4f08"
server_key_iv: "1236596a673e4806eabf2fff340f9431"
server_key_version: "encryptionkey@0"
I20260807 15:14:03.236331 13379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "80c9ad1daf924b26bde7cae22107964e"
format_stamp: "Formatted at 2026-08-07 15:14:03 on dist-test-slave-3kk6"
server_key: "f3dc4d3c0c0d2a6ef40788c3a92a4f08"
server_key_iv: "1236596a673e4806eabf2fff340f9431"
server_key_version: "encryptionkey@0"
I20260807 15:14:03.240454 13379 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 15:14:03.243124 13400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:03.244460 13379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:14:03.244616 13379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "80c9ad1daf924b26bde7cae22107964e"
format_stamp: "Formatted at 2026-08-07 15:14:03 on dist-test-slave-3kk6"
server_key: "f3dc4d3c0c0d2a6ef40788c3a92a4f08"
server_key_iv: "1236596a673e4806eabf2fff340f9431"
server_key_version: "encryptionkey@0"
I20260807 15:14:03.244750 13379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:03.259907 13379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:03.263942 13379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:03.264184 13379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:03.272703 13379 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32795
I20260807 15:14:03.272734 13472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32795 every 8 connection(s)
I20260807 15:14:03.273816 13379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:03.276804 13474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:03.277485 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 13379
I20260807 15:14:03.277640 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:14:03.277994 16265 external_mini_cluster.cc:1428] Setting key d9f6671626270044de2da2e983006522
I20260807 15:14:03.283581 13474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80c9ad1daf924b26bde7cae22107964e: Bootstrap starting.
I20260807 15:14:03.285979 13474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80c9ad1daf924b26bde7cae22107964e: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:03.286950 13474 log.cc:824] T 00000000000000000000000000000000 P 80c9ad1daf924b26bde7cae22107964e: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:03.288015 13478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:03.279621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52399 (local address 127.0.0.1:32795)
0807 15:14:03.280099 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:03.280115 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:03.280153 (+ 38us) server_negotiation.cc:407] Connection header received
0807 15:14:03.280829 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:03.280847 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:03.281210 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:03.281583 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:03.282015 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.283088 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.284113 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.284615 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.284802 (+ 187us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:03.285835 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:03.285841 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:03.285855 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:03.286330 (+ 475us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:03.286390 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:03.286415 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:03.286576 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:03.286824 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 15:14:03.286987 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":138,"threads_started":1}
I20260807 15:14:03.289335 13474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80c9ad1daf924b26bde7cae22107964e: No bootstrap required, opened a new log
I20260807 15:14:03.291359 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:14:03.293763 13474 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80c9ad1daf924b26bde7cae22107964e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80c9ad1daf924b26bde7cae22107964e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32795 } }
I20260807 15:14:03.294042 13474 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80c9ad1daf924b26bde7cae22107964e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:03.294167 13474 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80c9ad1daf924b26bde7cae22107964e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80c9ad1daf924b26bde7cae22107964e, State: Initialized, Role: FOLLOWER
I20260807 15:14:03.294780 13474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80c9ad1daf924b26bde7cae22107964e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80c9ad1daf924b26bde7cae22107964e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32795 } }
I20260807 15:14:03.294953 13474 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80c9ad1daf924b26bde7cae22107964e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:03.295045 13474 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80c9ad1daf924b26bde7cae22107964e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:03.295178 13474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80c9ad1daf924b26bde7cae22107964e [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:03.296502 13474 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80c9ad1daf924b26bde7cae22107964e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80c9ad1daf924b26bde7cae22107964e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32795 } }
I20260807 15:14:03.296924 13474 leader_election.cc:302] T 00000000000000000000000000000000 P 80c9ad1daf924b26bde7cae22107964e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80c9ad1daf924b26bde7cae22107964e; no voters:
I20260807 15:14:03.297330 13474 leader_election.cc:288] T 00000000000000000000000000000000 P 80c9ad1daf924b26bde7cae22107964e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:03.297413 13488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80c9ad1daf924b26bde7cae22107964e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:03.297644 13488 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80c9ad1daf924b26bde7cae22107964e [term 1 LEADER]: Becoming Leader. State: Replica: 80c9ad1daf924b26bde7cae22107964e, State: Running, Role: LEADER
I20260807 15:14:03.298027 13488 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80c9ad1daf924b26bde7cae22107964e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80c9ad1daf924b26bde7cae22107964e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32795 } }
I20260807 15:14:03.298620 13478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:03.293604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47997 (local address 127.0.0.1:32795)
0807 15:14:03.293734 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:03.293739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:03.293970 (+ 231us) server_negotiation.cc:407] Connection header received
0807 15:14:03.294156 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:03.294160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:03.294218 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:03.294323 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:03.294815 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.295574 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.296760 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:03.297034 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:03.297146 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:03.298131 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:03.298134 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:03.298137 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:03.298150 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:03.298179 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:03.298194 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:03.298283 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:03.298415 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 15:14:03.298470 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 15:14:03.298831 13474 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80c9ad1daf924b26bde7cae22107964e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:03.300559 13489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80c9ad1daf924b26bde7cae22107964e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80c9ad1daf924b26bde7cae22107964e. Latest consensus state: current_term: 1 leader_uuid: "80c9ad1daf924b26bde7cae22107964e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80c9ad1daf924b26bde7cae22107964e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32795 } } }
I20260807 15:14:03.300705 13489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80c9ad1daf924b26bde7cae22107964e [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:03.301077 13490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80c9ad1daf924b26bde7cae22107964e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80c9ad1daf924b26bde7cae22107964e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80c9ad1daf924b26bde7cae22107964e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32795 } } }
I20260807 15:14:03.301218 13490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80c9ad1daf924b26bde7cae22107964e [sys.catalog]: This master's current role is: LEADER
W20260807 15:14:03.304293 13511 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 80c9ad1daf924b26bde7cae22107964e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:14:03.304388 13511 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:14:03.304463 13504 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:03.305222 13504 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:03.311884 13504 catalog_manager.cc:1347] Generated new cluster ID: 696a3317b42d4dfc976a648fdae00e6e
I20260807 15:14:03.311990 13504 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:03.344753 13504 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:03.345999 13504 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:03.354759 13504 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80c9ad1daf924b26bde7cae22107964e: Generated new TSK 0
I20260807 15:14:03.355808 13504 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:14:03.360941 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 13379
W20260807 15:14:03.380187 13359 connection.cc:535] client connection to 127.0.0.1:32795 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32795): Connection reset by peer (error 104)
2026-08-07T15:14:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1683 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:03 dist-test-slave-3kk6 krb5kdc[13519](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:03 dist-test-slave-3kk6 krb5kdc[13519](info): set up 2 sockets
Aug 07 15:14:03 dist-test-slave-3kk6 krb5kdc[13519](info): commencing operation
krb5kdc: starting...
W20260807 15:14:04.617437 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:14:04 dist-test-slave-3kk6 krb5kdc[13519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:14:04.762487 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44509
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:04.881528 13533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:04.881922 13533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:04.882014 13533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:04.886229 13533 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:04.886358 13533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:04.886400 13533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:04.886442 13533 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:04.886479 13533 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:04.892225 13533 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44509
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:04.893476 13533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:04.894604 13533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:04.903949 13540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:04.903918 13544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:04.903918 13539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:04.904131 13533 server_base.cc:1034] running on GCE node
I20260807 15:14:04.904970 13533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:04.906230 13533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:04.907445 13533 hybrid_clock.cc:648] HybridClock initialized: now 1786115644907434 us; error 46 us; skew 500 ppm
Aug 07 15:14:04 dist-test-slave-3kk6 krb5kdc[13519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115644, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:04.910804 13533 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:14:04.912000 13533 webserver.cc:460] Webserver started at http://127.0.0.1:45571/ using document root <none> and password file <none>
I20260807 15:14:04.912725 13533 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:04.912839 13533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:04.913123 13533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:04.915745 13533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "1c870addf9944ef98b5d4325db724abd"
format_stamp: "Formatted at 2026-08-07 15:14:04 on dist-test-slave-3kk6"
server_key: "adfcc978af1928ae6b3d9c50b063e18c"
server_key_iv: "4c5840177389554d46a98e5e1328f9e7"
server_key_version: "encryptionkey@0"
I20260807 15:14:04.916635 13533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "1c870addf9944ef98b5d4325db724abd"
format_stamp: "Formatted at 2026-08-07 15:14:04 on dist-test-slave-3kk6"
server_key: "adfcc978af1928ae6b3d9c50b063e18c"
server_key_iv: "4c5840177389554d46a98e5e1328f9e7"
server_key_version: "encryptionkey@0"
I20260807 15:14:04.921617 13533 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:14:04.924829 13554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:04.926390 13533 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 15:14:04.926590 13533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "1c870addf9944ef98b5d4325db724abd"
format_stamp: "Formatted at 2026-08-07 15:14:04 on dist-test-slave-3kk6"
server_key: "adfcc978af1928ae6b3d9c50b063e18c"
server_key_iv: "4c5840177389554d46a98e5e1328f9e7"
server_key_version: "encryptionkey@0"
I20260807 15:14:04.926803 13533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:04.932142 13533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:04.936475 13533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:04.936734 13533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:04.944947 13533 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44509
I20260807 15:14:04.945014 13627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44509 every 8 connection(s)
I20260807 15:14:04.946120 13533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:04.949329 13628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:04.951196 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 13533
I20260807 15:14:04.951367 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:14:04.951719 16265 external_mini_cluster.cc:1428] Setting key 87d6e352853302844117b67a9a49cba6
I20260807 15:14:04.957018 13628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c870addf9944ef98b5d4325db724abd: Bootstrap starting.
I20260807 15:14:04.958158 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:04.953450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49737 (local address 127.0.0.1:44509)
0807 15:14:04.954028 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:04.954045 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:04.954089 (+ 44us) server_negotiation.cc:407] Connection header received
0807 15:14:04.954950 (+ 861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:04.954972 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:04.955297 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:04.955687 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:04.956734 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:04.956738 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:04.956753 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:04.956764 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:04.956870 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:04.956892 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:04.957064 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:04.957141 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 15:14:04.957324 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":180,"threads_started":1}
I20260807 15:14:04.959831 13628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c870addf9944ef98b5d4325db724abd: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:04.960878 13628 log.cc:824] T 00000000000000000000000000000000 P 1c870addf9944ef98b5d4325db724abd: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:04.961470 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:14:04.963729 13628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c870addf9944ef98b5d4325db724abd: No bootstrap required, opened a new log
I20260807 15:14:04.965951 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:04.964151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48459 (local address 127.0.0.1:44509)
0807 15:14:04.964298 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:04.964303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:04.964479 (+ 176us) server_negotiation.cc:407] Connection header received
0807 15:14:04.964575 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:04.964578 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:04.964615 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:04.964710 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:04.965551 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:04.965555 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:04.965558 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:04.965570 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:04.965601 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:04.965617 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:04.965711 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:04.965773 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 15:14:04.965813 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 15:14:04.968887 13628 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c870addf9944ef98b5d4325db724abd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c870addf9944ef98b5d4325db724abd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44509 } }
I20260807 15:14:04.969179 13628 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c870addf9944ef98b5d4325db724abd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:04.969218 13628 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c870addf9944ef98b5d4325db724abd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c870addf9944ef98b5d4325db724abd, State: Initialized, Role: FOLLOWER
I20260807 15:14:04.969681 13628 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c870addf9944ef98b5d4325db724abd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c870addf9944ef98b5d4325db724abd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44509 } }
I20260807 15:14:04.969861 13628 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c870addf9944ef98b5d4325db724abd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:04.969944 13628 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c870addf9944ef98b5d4325db724abd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:04.970046 13628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c870addf9944ef98b5d4325db724abd [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:04.971238 13628 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c870addf9944ef98b5d4325db724abd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c870addf9944ef98b5d4325db724abd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44509 } }
I20260807 15:14:04.971655 13628 leader_election.cc:302] T 00000000000000000000000000000000 P 1c870addf9944ef98b5d4325db724abd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c870addf9944ef98b5d4325db724abd; no voters:
I20260807 15:14:04.972025 13628 leader_election.cc:288] T 00000000000000000000000000000000 P 1c870addf9944ef98b5d4325db724abd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:04.972288 13642 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c870addf9944ef98b5d4325db724abd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:04.972580 13642 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c870addf9944ef98b5d4325db724abd [term 1 LEADER]: Becoming Leader. State: Replica: 1c870addf9944ef98b5d4325db724abd, State: Running, Role: LEADER
I20260807 15:14:04.973027 13642 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c870addf9944ef98b5d4325db724abd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c870addf9944ef98b5d4325db724abd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44509 } }
I20260807 15:14:04.973582 13628 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c870addf9944ef98b5d4325db724abd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:04.974964 13644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c870addf9944ef98b5d4325db724abd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c870addf9944ef98b5d4325db724abd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c870addf9944ef98b5d4325db724abd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44509 } } }
I20260807 15:14:04.975136 13644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c870addf9944ef98b5d4325db724abd [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:04.975438 13645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c870addf9944ef98b5d4325db724abd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c870addf9944ef98b5d4325db724abd. Latest consensus state: current_term: 1 leader_uuid: "1c870addf9944ef98b5d4325db724abd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c870addf9944ef98b5d4325db724abd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44509 } } }
I20260807 15:14:04.975598 13645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c870addf9944ef98b5d4325db724abd [sys.catalog]: This master's current role is: LEADER
W20260807 15:14:04.979066 13665 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1c870addf9944ef98b5d4325db724abd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:14:04.979199 13665 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:14:04.979285 13658 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:04.980033 13658 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:04.986260 13658 catalog_manager.cc:1347] Generated new cluster ID: 3735f952cf0541acacf56b1ed4d8ef33
I20260807 15:14:04.986377 13658 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:05.018836 13658 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:05.019228 13658 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:05.026499 13658 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c870addf9944ef98b5d4325db724abd: Generated new TSK 0
I20260807 15:14:05.027329 13658 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:14:05.031488 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 13533
2026-08-07T15:14:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1665 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:05 dist-test-slave-3kk6 krb5kdc[13671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:05 dist-test-slave-3kk6 krb5kdc[13671](info): set up 2 sockets
Aug 07 15:14:05 dist-test-slave-3kk6 krb5kdc[13671](info): commencing operation
krb5kdc: starting...
W20260807 15:14:06.309084 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:14:06 dist-test-slave-3kk6 krb5kdc[13671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:14:06.459117 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38861
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:06.585636 13693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:06.586100 13693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:06.586220 13693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:06.590603 13693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:06.590807 13693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:06.590849 13693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:06.590888 13693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:06.590924 13693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:06.596709 13693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:06.597959 13693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:06.599121 13693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:14:06.608557 13693 server_base.cc:1034] running on GCE node
W20260807 15:14:06.608439 13703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:06.608456 13699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:06.608440 13700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:06.609521 13693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:06.610898 13693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:06.612075 13693 hybrid_clock.cc:648] HybridClock initialized: now 1786115646612047 us; error 57 us; skew 500 ppm
Aug 07 15:14:06 dist-test-slave-3kk6 krb5kdc[13671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115646, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:06.620210 13693 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:14:06.621439 13693 webserver.cc:460] Webserver started at http://127.0.0.1:41825/ using document root <none> and password file <none>
I20260807 15:14:06.622092 13693 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:06.622169 13693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:06.622416 13693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:06.624857 13693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "c90fd15d04bf423e8ab7a8e0c41ba150"
format_stamp: "Formatted at 2026-08-07 15:14:06 on dist-test-slave-3kk6"
server_key: "e12eaefda3f24d3cf5ba72a84088b0d0"
server_key_iv: "223c5a53590090df3c234c9394c7d82a"
server_key_version: "encryptionkey@0"
I20260807 15:14:06.625545 13693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "c90fd15d04bf423e8ab7a8e0c41ba150"
format_stamp: "Formatted at 2026-08-07 15:14:06 on dist-test-slave-3kk6"
server_key: "e12eaefda3f24d3cf5ba72a84088b0d0"
server_key_iv: "223c5a53590090df3c234c9394c7d82a"
server_key_version: "encryptionkey@0"
I20260807 15:14:06.630048 13693 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260807 15:14:06.632887 13711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:06.634269 13693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:14:06.634431 13693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "c90fd15d04bf423e8ab7a8e0c41ba150"
format_stamp: "Formatted at 2026-08-07 15:14:06 on dist-test-slave-3kk6"
server_key: "e12eaefda3f24d3cf5ba72a84088b0d0"
server_key_iv: "223c5a53590090df3c234c9394c7d82a"
server_key_version: "encryptionkey@0"
I20260807 15:14:06.634565 13693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:06.648257 13693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:06.652052 13693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:06.652330 13693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:06.660709 13693 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38861
I20260807 15:14:06.660730 13798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38861 every 8 connection(s)
I20260807 15:14:06.661844 13693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:06.665040 13799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:06.668483 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 13693
I20260807 15:14:06.668663 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:14:06.669018 16265 external_mini_cluster.cc:1428] Setting key cb0484d789d86716df9058826aa29afa
I20260807 15:14:06.672298 13799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c90fd15d04bf423e8ab7a8e0c41ba150: Bootstrap starting.
I20260807 15:14:06.675210 13799 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c90fd15d04bf423e8ab7a8e0c41ba150: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:06.676088 13799 log.cc:824] T 00000000000000000000000000000000 P c90fd15d04bf423e8ab7a8e0c41ba150: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:06.678630 13799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c90fd15d04bf423e8ab7a8e0c41ba150: No bootstrap required, opened a new log
Aug 07 15:14:06 dist-test-slave-3kk6 krb5kdc[13671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 15:14:06.682410 13799 raft_consensus.cc:348] T 00000000000000000000000000000000 P c90fd15d04bf423e8ab7a8e0c41ba150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c90fd15d04bf423e8ab7a8e0c41ba150" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38861 } }
I20260807 15:14:06.682735 13799 raft_consensus.cc:374] T 00000000000000000000000000000000 P c90fd15d04bf423e8ab7a8e0c41ba150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:06.682863 13799 raft_consensus.cc:729] T 00000000000000000000000000000000 P c90fd15d04bf423e8ab7a8e0c41ba150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c90fd15d04bf423e8ab7a8e0c41ba150, State: Initialized, Role: FOLLOWER
I20260807 15:14:06.683375 13799 consensus_queue.cc:260] T 00000000000000000000000000000000 P c90fd15d04bf423e8ab7a8e0c41ba150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c90fd15d04bf423e8ab7a8e0c41ba150" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38861 } }
I20260807 15:14:06.683507 13799 raft_consensus.cc:388] T 00000000000000000000000000000000 P c90fd15d04bf423e8ab7a8e0c41ba150 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:06.683655 13799 raft_consensus.cc:482] T 00000000000000000000000000000000 P c90fd15d04bf423e8ab7a8e0c41ba150 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:06.683787 13799 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c90fd15d04bf423e8ab7a8e0c41ba150 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:06.684808 13799 raft_consensus.cc:504] T 00000000000000000000000000000000 P c90fd15d04bf423e8ab7a8e0c41ba150 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c90fd15d04bf423e8ab7a8e0c41ba150" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38861 } }
I20260807 15:14:06.685179 13799 leader_election.cc:302] T 00000000000000000000000000000000 P c90fd15d04bf423e8ab7a8e0c41ba150 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c90fd15d04bf423e8ab7a8e0c41ba150; no voters:
I20260807 15:14:06.685600 13799 leader_election.cc:288] T 00000000000000000000000000000000 P c90fd15d04bf423e8ab7a8e0c41ba150 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:06.685686 13806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c90fd15d04bf423e8ab7a8e0c41ba150 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:06.685973 13806 raft_consensus.cc:686] T 00000000000000000000000000000000 P c90fd15d04bf423e8ab7a8e0c41ba150 [term 1 LEADER]: Becoming Leader. State: Replica: c90fd15d04bf423e8ab7a8e0c41ba150, State: Running, Role: LEADER
I20260807 15:14:06.686311 13806 consensus_queue.cc:237] T 00000000000000000000000000000000 P c90fd15d04bf423e8ab7a8e0c41ba150 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c90fd15d04bf423e8ab7a8e0c41ba150" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38861 } }
I20260807 15:14:06.687166 13799 sys_catalog.cc:564] T 00000000000000000000000000000000 P c90fd15d04bf423e8ab7a8e0c41ba150 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:06.687227 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.670755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36373 (local address 127.0.0.1:38861)
0807 15:14:06.671264 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:06.671295 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:06.671343 (+ 48us) server_negotiation.cc:407] Connection header received
0807 15:14:06.672199 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:06.672223 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:06.672692 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:06.673078 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:06.673831 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.675390 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.676342 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.676937 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.677154 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.680613 (+ 3459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:06.680619 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:06.680634 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:06.680665 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:06.683468 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.684116 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.684123 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.684130 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.684264 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.684590 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.684594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.684596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.685101 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:06.685308 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:06.685634 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 15:14:06.685993 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":332,"thread_start_us":141,"threads_started":1}
I20260807 15:14:06.687716 13806 sys_catalog.cc:455] T 00000000000000000000000000000000 P c90fd15d04bf423e8ab7a8e0c41ba150 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c90fd15d04bf423e8ab7a8e0c41ba150. Latest consensus state: current_term: 1 leader_uuid: "c90fd15d04bf423e8ab7a8e0c41ba150" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c90fd15d04bf423e8ab7a8e0c41ba150" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38861 } } }
I20260807 15:14:06.687924 13806 sys_catalog.cc:458] T 00000000000000000000000000000000 P c90fd15d04bf423e8ab7a8e0c41ba150 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:06.688198 13806 sys_catalog.cc:455] T 00000000000000000000000000000000 P c90fd15d04bf423e8ab7a8e0c41ba150 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c90fd15d04bf423e8ab7a8e0c41ba150" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c90fd15d04bf423e8ab7a8e0c41ba150" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38861 } } }
I20260807 15:14:06.688372 13806 sys_catalog.cc:458] T 00000000000000000000000000000000 P c90fd15d04bf423e8ab7a8e0c41ba150 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:06.688798 13812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:06.690872 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:14:06.693209 13812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:06.698922 13812 catalog_manager.cc:1347] Generated new cluster ID: d8ddbbd6c06b418687f1a895af95eba9
I20260807 15:14:06.699011 13812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:06.703658 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:06.693295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37613 (local address 127.0.0.1:38861)
0807 15:14:06.693440 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:06.693446 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:06.693661 (+ 215us) server_negotiation.cc:407] Connection header received
0807 15:14:06.693861 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:06.693865 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:06.693920 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:06.693977 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:06.695069 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.695944 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.697205 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:06.697578 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:06.697662 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:06.699926 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:06.699929 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:06.699932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:06.699959 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:06.701768 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.702324 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.702328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.702330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.702417 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:06.702750 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:06.702753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:06.702754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:06.702963 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:06.703086 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:06.703374 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 15:14:06.703498 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 15:14:06.709830 13812 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:06.711229 13812 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:06.720551 13812 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c90fd15d04bf423e8ab7a8e0c41ba150: Generated new TSK 0
I20260807 15:14:06.721514 13812 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:14:06.730652 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 13693
2026-08-07T15:14:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1700 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:06 dist-test-slave-3kk6 krb5kdc[13837](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:06 dist-test-slave-3kk6 krb5kdc[13837](info): set up 2 sockets
Aug 07 15:14:06 dist-test-slave-3kk6 krb5kdc[13837](info): commencing operation
krb5kdc: starting...
W20260807 15:14:07.972846 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:14:08 dist-test-slave-3kk6 krb5kdc[13837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115648, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:14:08.118158 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44847
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:08.241160 13853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:08.241493 13853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:08.241539 13853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:08.245709 13853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:08.245801 13853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:08.245823 13853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:08.245841 13853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:08.245857 13853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:08.251574 13853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:08.252784 13853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:08.253831 13853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:08.263175 13859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:08.263195 13862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:08.263175 13860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:08.263216 13853 server_base.cc:1034] running on GCE node
I20260807 15:14:08.264181 13853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:08.265386 13853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:08.266587 13853 hybrid_clock.cc:648] HybridClock initialized: now 1786115648266555 us; error 47 us; skew 500 ppm
Aug 07 15:14:08 dist-test-slave-3kk6 krb5kdc[13837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115648, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:08.269764 13853 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:14:08.270812 13853 webserver.cc:460] Webserver started at http://127.0.0.1:39683/ using document root <none> and password file <none>
I20260807 15:14:08.271381 13853 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:08.271454 13853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:08.271636 13853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:08.273849 13853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "79d9085a811a42468d22e661aa039555"
format_stamp: "Formatted at 2026-08-07 15:14:08 on dist-test-slave-3kk6"
server_key: "edcaa7f7575a32976268ab86fd2b72cf"
server_key_iv: "0291e2a47ae69b9c9f7ab884de12023b"
server_key_version: "encryptionkey@0"
I20260807 15:14:08.274451 13853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "79d9085a811a42468d22e661aa039555"
format_stamp: "Formatted at 2026-08-07 15:14:08 on dist-test-slave-3kk6"
server_key: "edcaa7f7575a32976268ab86fd2b72cf"
server_key_iv: "0291e2a47ae69b9c9f7ab884de12023b"
server_key_version: "encryptionkey@0"
I20260807 15:14:08.278635 13853 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 15:14:08.281355 13871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:08.282853 13853 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 15:14:08.283003 13853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "79d9085a811a42468d22e661aa039555"
format_stamp: "Formatted at 2026-08-07 15:14:08 on dist-test-slave-3kk6"
server_key: "edcaa7f7575a32976268ab86fd2b72cf"
server_key_iv: "0291e2a47ae69b9c9f7ab884de12023b"
server_key_version: "encryptionkey@0"
I20260807 15:14:08.283134 13853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:08.310099 13853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:08.313668 13853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:08.313910 13853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:08.321677 13853 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44847
I20260807 15:14:08.321750 13939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44847 every 8 connection(s)
I20260807 15:14:08.322825 13853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:08.325939 13945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:08.327414 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 13853
I20260807 15:14:08.327586 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:14:08.327916 16265 external_mini_cluster.cc:1428] Setting key c7e08ddd7d7018bd484281acd70158e5
I20260807 15:14:08.333449 13945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79d9085a811a42468d22e661aa039555: Bootstrap starting.
I20260807 15:14:08.336397 13945 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79d9085a811a42468d22e661aa039555: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:08.337452 13945 log.cc:824] T 00000000000000000000000000000000 P 79d9085a811a42468d22e661aa039555: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:08.338971 13950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:08.329527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38071 (local address 127.0.0.1:44847)
0807 15:14:08.330002 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:08.330024 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:08.330196 (+ 172us) server_negotiation.cc:407] Connection header received
0807 15:14:08.331128 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:08.331155 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:08.331624 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:08.332037 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:08.332513 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:08.333959 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:08.334982 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:08.335685 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:08.335856 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:08.336838 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:08.336844 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:08.336862 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:08.337416 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:08.337492 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:08.337525 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:08.337636 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:08.337782 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 15:14:08.338016 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":114,"threads_started":1}
I20260807 15:14:08.340058 13945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79d9085a811a42468d22e661aa039555: No bootstrap required, opened a new log
I20260807 15:14:08.342101 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:14:08.344354 13945 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79d9085a811a42468d22e661aa039555 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d9085a811a42468d22e661aa039555" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44847 } }
I20260807 15:14:08.344719 13945 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79d9085a811a42468d22e661aa039555 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:08.344794 13945 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79d9085a811a42468d22e661aa039555 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79d9085a811a42468d22e661aa039555, State: Initialized, Role: FOLLOWER
I20260807 15:14:08.345376 13945 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79d9085a811a42468d22e661aa039555 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d9085a811a42468d22e661aa039555" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44847 } }
I20260807 15:14:08.345542 13945 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79d9085a811a42468d22e661aa039555 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:08.345638 13945 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79d9085a811a42468d22e661aa039555 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:08.345774 13945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79d9085a811a42468d22e661aa039555 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:08.347198 13945 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79d9085a811a42468d22e661aa039555 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d9085a811a42468d22e661aa039555" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44847 } }
I20260807 15:14:08.347612 13945 leader_election.cc:302] T 00000000000000000000000000000000 P 79d9085a811a42468d22e661aa039555 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79d9085a811a42468d22e661aa039555; no voters:
I20260807 15:14:08.348102 13945 leader_election.cc:288] T 00000000000000000000000000000000 P 79d9085a811a42468d22e661aa039555 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:08.348233 13959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79d9085a811a42468d22e661aa039555 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:08.348517 13959 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79d9085a811a42468d22e661aa039555 [term 1 LEADER]: Becoming Leader. State: Replica: 79d9085a811a42468d22e661aa039555, State: Running, Role: LEADER
I20260807 15:14:08.348968 13959 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79d9085a811a42468d22e661aa039555 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d9085a811a42468d22e661aa039555" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44847 } }
I20260807 15:14:08.349481 13950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:08.344439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59839 (local address 127.0.0.1:44847)
0807 15:14:08.344582 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:08.344586 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:08.344733 (+ 147us) server_negotiation.cc:407] Connection header received
0807 15:14:08.344889 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:08.344896 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:08.344958 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:08.345028 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:08.345369 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:08.346094 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:08.347347 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:08.347633 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:08.347724 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:08.348746 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:08.348750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:08.348753 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:08.348768 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:08.348799 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:08.348813 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:08.348884 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:08.349288 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 15:14:08.349350 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 15:14:08.349682 13945 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79d9085a811a42468d22e661aa039555 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:08.350478 13959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d9085a811a42468d22e661aa039555 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79d9085a811a42468d22e661aa039555. Latest consensus state: current_term: 1 leader_uuid: "79d9085a811a42468d22e661aa039555" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d9085a811a42468d22e661aa039555" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44847 } } }
I20260807 15:14:08.350706 13959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d9085a811a42468d22e661aa039555 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:08.351009 13960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d9085a811a42468d22e661aa039555 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79d9085a811a42468d22e661aa039555" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d9085a811a42468d22e661aa039555" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44847 } } }
I20260807 15:14:08.351130 13960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d9085a811a42468d22e661aa039555 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:08.351157 13970 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:08.354712 13970 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:08.361537 13970 catalog_manager.cc:1347] Generated new cluster ID: dfbbd8aa69f347c8a1378d43f6432115
I20260807 15:14:08.361655 13970 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:08.374101 13970 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:08.375361 13970 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:08.384858 13970 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79d9085a811a42468d22e661aa039555: Generated new TSK 0
I20260807 15:14:08.385684 13970 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:14:08.390084 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 13853
2026-08-07T15:14:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1661 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:08 dist-test-slave-3kk6 krb5kdc[13983](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:08 dist-test-slave-3kk6 krb5kdc[13983](info): set up 2 sockets
Aug 07 15:14:08 dist-test-slave-3kk6 krb5kdc[13983](info): commencing operation
krb5kdc: starting...
W20260807 15:14:09.678443 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:14:09 dist-test-slave-3kk6 krb5kdc[13983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:14:09.818424 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39811
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:09.939931 14000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:09.940299 14000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:09.940389 14000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:09.944680 14000 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:09.944831 14000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:09.944873 14000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:09.944931 14000 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:09.944988 14000 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:09.950938 14000 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:09.952272 14000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:09.953461 14000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:14:09.963529 14000 server_base.cc:1034] running on GCE node
W20260807 15:14:09.963584 14006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:09.963584 14005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:09.963754 14009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:09.964478 14000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:09.965790 14000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:09.966990 14000 hybrid_clock.cc:648] HybridClock initialized: now 1786115649966976 us; error 36 us; skew 500 ppm
Aug 07 15:14:09 dist-test-slave-3kk6 krb5kdc[13983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115649, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:09.970300 14000 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:14:09.971446 14000 webserver.cc:460] Webserver started at http://127.0.0.1:44365/ using document root <none> and password file <none>
I20260807 15:14:09.972074 14000 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:09.972147 14000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:09.972343 14000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:09.974750 14000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "4850af76b8d24a9da61fcf199cb35314"
format_stamp: "Formatted at 2026-08-07 15:14:09 on dist-test-slave-3kk6"
server_key: "6e31760a6459905f9c23537301486082"
server_key_iv: "b58d9c86f813c2b8d12e13842efaef1f"
server_key_version: "encryptionkey@0"
I20260807 15:14:09.975394 14000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "4850af76b8d24a9da61fcf199cb35314"
format_stamp: "Formatted at 2026-08-07 15:14:09 on dist-test-slave-3kk6"
server_key: "6e31760a6459905f9c23537301486082"
server_key_iv: "b58d9c86f813c2b8d12e13842efaef1f"
server_key_version: "encryptionkey@0"
I20260807 15:14:09.979892 14000 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:14:09.982903 14019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:09.984423 14000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 15:14:09.984584 14000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "4850af76b8d24a9da61fcf199cb35314"
format_stamp: "Formatted at 2026-08-07 15:14:09 on dist-test-slave-3kk6"
server_key: "6e31760a6459905f9c23537301486082"
server_key_iv: "b58d9c86f813c2b8d12e13842efaef1f"
server_key_version: "encryptionkey@0"
I20260807 15:14:09.984724 14000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:09.989988 14000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:09.993443 14000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:09.993695 14000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:10.001688 14000 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39811
I20260807 15:14:10.001701 14090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39811 every 8 connection(s)
I20260807 15:14:10.002856 14000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:10.006105 14091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:10.007463 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 14000
I20260807 15:14:10.007624 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:14:10.007975 16265 external_mini_cluster.cc:1428] Setting key 441b5c204e73ba75b60979592b624aa8
I20260807 15:14:10.013554 14091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4850af76b8d24a9da61fcf199cb35314: Bootstrap starting.
I20260807 15:14:10.014031 14100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:10.009649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53165 (local address 127.0.0.1:39811)
0807 15:14:10.010161 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:10.010177 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:10.010220 (+ 43us) server_negotiation.cc:407] Connection header received
0807 15:14:10.011009 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:10.011030 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:10.011346 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:10.011738 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:10.012688 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:10.012694 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:10.012706 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:10.012717 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:10.012790 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:10.012811 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:10.012989 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:10.013065 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 15:14:10.013234 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":150,"threads_started":1}
I20260807 15:14:10.016288 14091 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4850af76b8d24a9da61fcf199cb35314: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:10.017234 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:14:10.017256 14091 log.cc:824] T 00000000000000000000000000000000 P 4850af76b8d24a9da61fcf199cb35314: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:10.019570 14091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4850af76b8d24a9da61fcf199cb35314: No bootstrap required, opened a new log
I20260807 15:14:10.021294 14100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:10.019474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55555 (local address 127.0.0.1:39811)
0807 15:14:10.019623 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:10.019627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:10.019864 (+ 237us) server_negotiation.cc:407] Connection header received
0807 15:14:10.019986 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:10.019988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:10.020021 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:10.020091 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:10.020863 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:10.020866 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:10.020868 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:10.020879 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:10.020905 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:10.020915 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:10.021011 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:10.021124 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 15:14:10.021158 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 15:14:10.024423 14091 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4850af76b8d24a9da61fcf199cb35314 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4850af76b8d24a9da61fcf199cb35314" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39811 } }
I20260807 15:14:10.024718 14091 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4850af76b8d24a9da61fcf199cb35314 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:10.024787 14091 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4850af76b8d24a9da61fcf199cb35314 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4850af76b8d24a9da61fcf199cb35314, State: Initialized, Role: FOLLOWER
I20260807 15:14:10.025301 14091 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4850af76b8d24a9da61fcf199cb35314 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4850af76b8d24a9da61fcf199cb35314" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39811 } }
I20260807 15:14:10.025501 14091 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4850af76b8d24a9da61fcf199cb35314 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:10.025580 14091 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4850af76b8d24a9da61fcf199cb35314 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:10.025728 14091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4850af76b8d24a9da61fcf199cb35314 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:10.026865 14091 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4850af76b8d24a9da61fcf199cb35314 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4850af76b8d24a9da61fcf199cb35314" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39811 } }
I20260807 15:14:10.027266 14091 leader_election.cc:302] T 00000000000000000000000000000000 P 4850af76b8d24a9da61fcf199cb35314 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4850af76b8d24a9da61fcf199cb35314; no voters:
I20260807 15:14:10.027644 14091 leader_election.cc:288] T 00000000000000000000000000000000 P 4850af76b8d24a9da61fcf199cb35314 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:10.027863 14109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4850af76b8d24a9da61fcf199cb35314 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:10.028180 14109 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4850af76b8d24a9da61fcf199cb35314 [term 1 LEADER]: Becoming Leader. State: Replica: 4850af76b8d24a9da61fcf199cb35314, State: Running, Role: LEADER
I20260807 15:14:10.028518 14109 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4850af76b8d24a9da61fcf199cb35314 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4850af76b8d24a9da61fcf199cb35314" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39811 } }
I20260807 15:14:10.029034 14091 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4850af76b8d24a9da61fcf199cb35314 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:10.030310 14110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4850af76b8d24a9da61fcf199cb35314 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4850af76b8d24a9da61fcf199cb35314" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4850af76b8d24a9da61fcf199cb35314" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39811 } } }
I20260807 15:14:10.030457 14110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4850af76b8d24a9da61fcf199cb35314 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:10.030901 14122 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:10.030872 14111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4850af76b8d24a9da61fcf199cb35314 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4850af76b8d24a9da61fcf199cb35314. Latest consensus state: current_term: 1 leader_uuid: "4850af76b8d24a9da61fcf199cb35314" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4850af76b8d24a9da61fcf199cb35314" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39811 } } }
I20260807 15:14:10.031145 14111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4850af76b8d24a9da61fcf199cb35314 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:10.034333 14122 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:10.040892 14122 catalog_manager.cc:1347] Generated new cluster ID: b07fad0afd8449a19295d0c7ecd861cf
I20260807 15:14:10.041069 14122 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:10.058918 14122 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:10.059324 14122 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:10.074026 14122 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4850af76b8d24a9da61fcf199cb35314: Generated new TSK 0
I20260807 15:14:10.075052 14122 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:14:10.080658 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 14000
2026-08-07T15:14:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1686 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:10 dist-test-slave-3kk6 krb5kdc[14137](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:10 dist-test-slave-3kk6 krb5kdc[14137](info): set up 2 sockets
Aug 07 15:14:10 dist-test-slave-3kk6 krb5kdc[14137](info): commencing operation
krb5kdc: starting...
W20260807 15:14:11.320128 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:14:11 dist-test-slave-3kk6 krb5kdc[14137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:14:11.462790 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33863
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:11.584421 14157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:11.584729 14157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:11.584775 14157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:11.588945 14157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:11.589027 14157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:11.589047 14157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:11.589066 14157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:11.589080 14157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:11.594789 14157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:11.596210 14157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:11.597461 14157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:14:11.607095 14157 server_base.cc:1034] running on GCE node
W20260807 15:14:11.607050 14163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:11.607050 14166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:11.607059 14164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:11.607957 14157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:11.609233 14157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:11.610378 14157 hybrid_clock.cc:648] HybridClock initialized: now 1786115651610351 us; error 45 us; skew 500 ppm
Aug 07 15:14:11 dist-test-slave-3kk6 krb5kdc[14137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115651, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:11.613574 14157 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:14:11.614553 14157 webserver.cc:460] Webserver started at http://127.0.0.1:38819/ using document root <none> and password file <none>
I20260807 15:14:11.615181 14157 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:11.615257 14157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:11.615442 14157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:11.617565 14157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "0fe416410fd84fb6a8ac5ef0067f6eef"
format_stamp: "Formatted at 2026-08-07 15:14:11 on dist-test-slave-3kk6"
server_key: "2b406a34c571587b4123b30151d4c21d"
server_key_iv: "d932875f5b6f3de56eacbab1b6c7e4cb"
server_key_version: "encryptionkey@0"
I20260807 15:14:11.618141 14157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "0fe416410fd84fb6a8ac5ef0067f6eef"
format_stamp: "Formatted at 2026-08-07 15:14:11 on dist-test-slave-3kk6"
server_key: "2b406a34c571587b4123b30151d4c21d"
server_key_iv: "d932875f5b6f3de56eacbab1b6c7e4cb"
server_key_version: "encryptionkey@0"
I20260807 15:14:11.622195 14157 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:14:11.624881 14174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:11.626209 14157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 15:14:11.626325 14157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "0fe416410fd84fb6a8ac5ef0067f6eef"
format_stamp: "Formatted at 2026-08-07 15:14:11 on dist-test-slave-3kk6"
server_key: "2b406a34c571587b4123b30151d4c21d"
server_key_iv: "d932875f5b6f3de56eacbab1b6c7e4cb"
server_key_version: "encryptionkey@0"
I20260807 15:14:11.626430 14157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:11.659175 14157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:11.663008 14157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:11.663211 14157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:11.671082 14157 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33863
I20260807 15:14:11.671110 14243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33863 every 8 connection(s)
I20260807 15:14:11.672219 14157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:11.675274 14244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:11.681648 14244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fe416410fd84fb6a8ac5ef0067f6eef: Bootstrap starting.
I20260807 15:14:11.682494 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 14157
I20260807 15:14:11.682695 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:14:11.683053 16265 external_mini_cluster.cc:1428] Setting key 016a401eef5b72516b09992b7bfee837
I20260807 15:14:11.685032 14244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fe416410fd84fb6a8ac5ef0067f6eef: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:11.685833 14244 log.cc:824] T 00000000000000000000000000000000 P 0fe416410fd84fb6a8ac5ef0067f6eef: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:11.688284 14244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fe416410fd84fb6a8ac5ef0067f6eef: No bootstrap required, opened a new log
Aug 07 15:14:11 dist-test-slave-3kk6 krb5kdc[14137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 15:14:11.693516 14244 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fe416410fd84fb6a8ac5ef0067f6eef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe416410fd84fb6a8ac5ef0067f6eef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33863 } }
I20260807 15:14:11.693895 14244 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fe416410fd84fb6a8ac5ef0067f6eef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:11.693965 14244 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fe416410fd84fb6a8ac5ef0067f6eef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fe416410fd84fb6a8ac5ef0067f6eef, State: Initialized, Role: FOLLOWER
I20260807 15:14:11.694633 14244 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fe416410fd84fb6a8ac5ef0067f6eef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe416410fd84fb6a8ac5ef0067f6eef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33863 } }
I20260807 15:14:11.694868 14244 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fe416410fd84fb6a8ac5ef0067f6eef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:11.694958 14244 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fe416410fd84fb6a8ac5ef0067f6eef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:11.695082 14244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fe416410fd84fb6a8ac5ef0067f6eef [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:11.696533 14244 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fe416410fd84fb6a8ac5ef0067f6eef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe416410fd84fb6a8ac5ef0067f6eef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33863 } }
I20260807 15:14:11.696947 14244 leader_election.cc:302] T 00000000000000000000000000000000 P 0fe416410fd84fb6a8ac5ef0067f6eef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fe416410fd84fb6a8ac5ef0067f6eef; no voters:
I20260807 15:14:11.697400 14244 leader_election.cc:288] T 00000000000000000000000000000000 P 0fe416410fd84fb6a8ac5ef0067f6eef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:11.697506 14251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fe416410fd84fb6a8ac5ef0067f6eef [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:11.697780 14251 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fe416410fd84fb6a8ac5ef0067f6eef [term 1 LEADER]: Becoming Leader. State: Replica: 0fe416410fd84fb6a8ac5ef0067f6eef, State: Running, Role: LEADER
I20260807 15:14:11.698210 14251 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fe416410fd84fb6a8ac5ef0067f6eef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe416410fd84fb6a8ac5ef0067f6eef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33863 } }
I20260807 15:14:11.699122 14244 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fe416410fd84fb6a8ac5ef0067f6eef [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:11.699676 14255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fe416410fd84fb6a8ac5ef0067f6eef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fe416410fd84fb6a8ac5ef0067f6eef. Latest consensus state: current_term: 1 leader_uuid: "0fe416410fd84fb6a8ac5ef0067f6eef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe416410fd84fb6a8ac5ef0067f6eef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33863 } } }
I20260807 15:14:11.699810 14255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fe416410fd84fb6a8ac5ef0067f6eef [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:11.700519 14248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:11.684467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:60313 (local address 127.0.0.1:33863)
0807 15:14:11.684965 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:11.684982 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:11.685019 (+ 37us) server_negotiation.cc:407] Connection header received
0807 15:14:11.685772 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:11.685791 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:11.686237 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:11.686634 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:11.687648 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.688742 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.689973 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.690473 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.690610 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:11.693958 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:11.693966 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:11.693988 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:11.694042 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:11.696455 (+ 2413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:11.697189 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:11.697197 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:11.697205 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:11.697385 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:11.697837 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:11.697841 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:11.697843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:11.698418 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:11.698713 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:11.699100 (+ 387us) server_negotiation.cc:299] Negotiation successful
0807 15:14:11.699376 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":156,"threads_started":1}
I20260807 15:14:11.700592 14258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fe416410fd84fb6a8ac5ef0067f6eef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fe416410fd84fb6a8ac5ef0067f6eef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe416410fd84fb6a8ac5ef0067f6eef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33863 } } }
I20260807 15:14:11.700695 14258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fe416410fd84fb6a8ac5ef0067f6eef [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:11.704279 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260807 15:14:11.705246 14274 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0fe416410fd84fb6a8ac5ef0067f6eef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:14:11.705336 14274 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:14:11.705405 14260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:11.706261 14260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:11.714782 14260 catalog_manager.cc:1347] Generated new cluster ID: a80438625fc249c8bee313d4f125745d
I20260807 15:14:11.714864 14260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:11.718978 14248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:11.708916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:39869 (local address 127.0.0.1:33863)
0807 15:14:11.709040 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:11.709045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:11.709329 (+ 284us) server_negotiation.cc:407] Connection header received
0807 15:14:11.709464 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:11.709467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:11.709520 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:11.709614 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:11.710504 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.711455 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.712691 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:11.712945 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:11.713026 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:11.715007 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:11.715012 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:11.715015 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:11.715043 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:11.716928 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:11.717518 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:11.717522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:11.717524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:11.717615 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:11.717967 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:11.717974 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:11.717975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:11.718200 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:11.718352 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:11.718645 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 15:14:11.718798 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 15:14:11.734508 14260 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:11.735549 14260 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:11.741351 14260 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fe416410fd84fb6a8ac5ef0067f6eef: Generated new TSK 0
I20260807 15:14:11.742228 14260 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:14:11.747942 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 14157
W20260807 15:14:11.760118 14135 connection.cc:535] client connection to 127.0.0.1:33863 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33863): Connection reset by peer (error 104)
2026-08-07T15:14:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1666 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:11 dist-test-slave-3kk6 krb5kdc[14290](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:11 dist-test-slave-3kk6 krb5kdc[14290](info): set up 2 sockets
Aug 07 15:14:11 dist-test-slave-3kk6 krb5kdc[14290](info): commencing operation
krb5kdc: starting...
W20260807 15:14:13.008673 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:14:13 dist-test-slave-3kk6 krb5kdc[14290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:14:13.155128 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41325
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:13.274693 14309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:13.274994 14309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:13.275043 14309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:13.279383 14309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:13.279469 14309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:13.279488 14309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:13.279507 14309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:13.279523 14309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:13.285039 14309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:13.286197 14309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:13.287371 14309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:13.296314 14320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:13.296489 14309 server_base.cc:1034] running on GCE node
W20260807 15:14:13.296334 14319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:13.296384 14322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:13.297257 14309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:13.298460 14309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:13.299623 14309 hybrid_clock.cc:648] HybridClock initialized: now 1786115653299597 us; error 50 us; skew 500 ppm
Aug 07 15:14:13 dist-test-slave-3kk6 krb5kdc[14290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115653, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:13.302563 14309 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:14:13.303527 14309 webserver.cc:460] Webserver started at http://127.0.0.1:44253/ using document root <none> and password file <none>
I20260807 15:14:13.304121 14309 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:13.304217 14309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:13.304476 14309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:13.306835 14309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "c048ae8c86aa479a93b2fdb6c0dcd9c6"
format_stamp: "Formatted at 2026-08-07 15:14:13 on dist-test-slave-3kk6"
server_key: "992b238c4f430f8f50d2e987d9d69291"
server_key_iv: "680ff0bd122bb50bcfdc9ee1241f53de"
server_key_version: "encryptionkey@0"
I20260807 15:14:13.307451 14309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "c048ae8c86aa479a93b2fdb6c0dcd9c6"
format_stamp: "Formatted at 2026-08-07 15:14:13 on dist-test-slave-3kk6"
server_key: "992b238c4f430f8f50d2e987d9d69291"
server_key_iv: "680ff0bd122bb50bcfdc9ee1241f53de"
server_key_version: "encryptionkey@0"
I20260807 15:14:13.311316 14309 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 15:14:13.314013 14329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:13.315392 14309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:14:13.315548 14309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "c048ae8c86aa479a93b2fdb6c0dcd9c6"
format_stamp: "Formatted at 2026-08-07 15:14:13 on dist-test-slave-3kk6"
server_key: "992b238c4f430f8f50d2e987d9d69291"
server_key_iv: "680ff0bd122bb50bcfdc9ee1241f53de"
server_key_version: "encryptionkey@0"
I20260807 15:14:13.315681 14309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:13.328886 14309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:13.332319 14309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:13.332571 14309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:13.340540 14309 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41325
I20260807 15:14:13.340567 14403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41325 every 8 connection(s)
I20260807 15:14:13.341574 14309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:13.343391 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 14309
I20260807 15:14:13.343545 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:14:13.343896 16265 external_mini_cluster.cc:1428] Setting key b30109a6656925a57af8c3adf3fcb8bb
I20260807 15:14:13.344726 14405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:13.350775 14405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c048ae8c86aa479a93b2fdb6c0dcd9c6: Bootstrap starting.
I20260807 15:14:13.353240 14405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c048ae8c86aa479a93b2fdb6c0dcd9c6: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:13.353258 14408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:13.345418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:59833 (local address 127.0.0.1:41325)
0807 15:14:13.345929 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:13.345945 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:13.345981 (+ 36us) server_negotiation.cc:407] Connection header received
0807 15:14:13.346702 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:13.346728 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:13.347096 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:13.347441 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:13.347873 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:13.348855 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:13.349787 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:13.350272 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:13.350450 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:13.351503 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:13.351508 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:13.351520 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:13.351968 (+ 448us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:13.352032 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:13.352055 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:13.352202 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:13.352391 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 15:14:13.352535 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":151,"threads_started":1}
I20260807 15:14:13.354054 14405 log.cc:824] T 00000000000000000000000000000000 P c048ae8c86aa479a93b2fdb6c0dcd9c6: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:13.356601 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:14:13.356609 14405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c048ae8c86aa479a93b2fdb6c0dcd9c6: No bootstrap required, opened a new log
I20260807 15:14:13.361207 14405 raft_consensus.cc:348] T 00000000000000000000000000000000 P c048ae8c86aa479a93b2fdb6c0dcd9c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c048ae8c86aa479a93b2fdb6c0dcd9c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41325 } }
I20260807 15:14:13.361552 14405 raft_consensus.cc:374] T 00000000000000000000000000000000 P c048ae8c86aa479a93b2fdb6c0dcd9c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:13.361698 14405 raft_consensus.cc:729] T 00000000000000000000000000000000 P c048ae8c86aa479a93b2fdb6c0dcd9c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c048ae8c86aa479a93b2fdb6c0dcd9c6, State: Initialized, Role: FOLLOWER
I20260807 15:14:13.362237 14405 consensus_queue.cc:260] T 00000000000000000000000000000000 P c048ae8c86aa479a93b2fdb6c0dcd9c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c048ae8c86aa479a93b2fdb6c0dcd9c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41325 } }
I20260807 15:14:13.362444 14405 raft_consensus.cc:388] T 00000000000000000000000000000000 P c048ae8c86aa479a93b2fdb6c0dcd9c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:13.362531 14405 raft_consensus.cc:482] T 00000000000000000000000000000000 P c048ae8c86aa479a93b2fdb6c0dcd9c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:13.362671 14405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c048ae8c86aa479a93b2fdb6c0dcd9c6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:13.363238 14408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:13.358781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:44075 (local address 127.0.0.1:41325)
0807 15:14:13.358945 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:13.358952 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:13.359063 (+ 111us) server_negotiation.cc:407] Connection header received
0807 15:14:13.359164 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:13.359167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:13.359216 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:13.359300 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:13.359553 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:13.360363 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:13.361188 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:13.361535 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:13.361650 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:13.362713 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:13.362717 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:13.362719 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:13.362737 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:13.362770 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:13.362784 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:13.362898 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:13.363045 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 15:14:13.363096 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 15:14:13.363978 14405 raft_consensus.cc:504] T 00000000000000000000000000000000 P c048ae8c86aa479a93b2fdb6c0dcd9c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c048ae8c86aa479a93b2fdb6c0dcd9c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41325 } }
I20260807 15:14:13.364357 14405 leader_election.cc:302] T 00000000000000000000000000000000 P c048ae8c86aa479a93b2fdb6c0dcd9c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c048ae8c86aa479a93b2fdb6c0dcd9c6; no voters:
I20260807 15:14:13.364722 14405 leader_election.cc:288] T 00000000000000000000000000000000 P c048ae8c86aa479a93b2fdb6c0dcd9c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:13.364825 14418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c048ae8c86aa479a93b2fdb6c0dcd9c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:13.365091 14418 raft_consensus.cc:686] T 00000000000000000000000000000000 P c048ae8c86aa479a93b2fdb6c0dcd9c6 [term 1 LEADER]: Becoming Leader. State: Replica: c048ae8c86aa479a93b2fdb6c0dcd9c6, State: Running, Role: LEADER
I20260807 15:14:13.365465 14418 consensus_queue.cc:237] T 00000000000000000000000000000000 P c048ae8c86aa479a93b2fdb6c0dcd9c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c048ae8c86aa479a93b2fdb6c0dcd9c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41325 } }
I20260807 15:14:13.366076 14405 sys_catalog.cc:564] T 00000000000000000000000000000000 P c048ae8c86aa479a93b2fdb6c0dcd9c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:13.367290 14420 sys_catalog.cc:455] T 00000000000000000000000000000000 P c048ae8c86aa479a93b2fdb6c0dcd9c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c048ae8c86aa479a93b2fdb6c0dcd9c6. Latest consensus state: current_term: 1 leader_uuid: "c048ae8c86aa479a93b2fdb6c0dcd9c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c048ae8c86aa479a93b2fdb6c0dcd9c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41325 } } }
I20260807 15:14:13.367458 14420 sys_catalog.cc:458] T 00000000000000000000000000000000 P c048ae8c86aa479a93b2fdb6c0dcd9c6 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:13.367290 14419 sys_catalog.cc:455] T 00000000000000000000000000000000 P c048ae8c86aa479a93b2fdb6c0dcd9c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c048ae8c86aa479a93b2fdb6c0dcd9c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c048ae8c86aa479a93b2fdb6c0dcd9c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41325 } } }
I20260807 15:14:13.367564 14419 sys_catalog.cc:458] T 00000000000000000000000000000000 P c048ae8c86aa479a93b2fdb6c0dcd9c6 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:13.367993 14430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:13.371217 14430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:13.376909 14430 catalog_manager.cc:1347] Generated new cluster ID: 32095bd5c7184e8eabb467c07b51ffa2
I20260807 15:14:13.376997 14430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:13.387177 14430 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:13.388291 14430 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:13.394359 14430 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c048ae8c86aa479a93b2fdb6c0dcd9c6: Generated new TSK 0
I20260807 15:14:13.395185 14430 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:14:13.403954 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 14309
W20260807 15:14:13.420363 14286 connection.cc:535] client connection to 127.0.0.1:41325 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41325): Connection reset by peer (error 104)
2026-08-07T15:14:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1660 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:13 dist-test-slave-3kk6 krb5kdc[14448](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:13 dist-test-slave-3kk6 krb5kdc[14448](info): set up 2 sockets
Aug 07 15:14:13 dist-test-slave-3kk6 krb5kdc[14448](info): commencing operation
krb5kdc: starting...
W20260807 15:14:14.657697 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:14:14 dist-test-slave-3kk6 krb5kdc[14448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:14:14.802431 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45569
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45569
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:14.920709 14472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:14.921080 14472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:14.921170 14472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:14.925505 14472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:14.925637 14472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:14.925674 14472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:14.925714 14472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:14.925747 14472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:14.931442 14472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:14.932694 14472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:14.933826 14472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:14.943300 14481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:14.943338 14483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:14.943302 14480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:14.943552 14472 server_base.cc:1034] running on GCE node
I20260807 15:14:14.944294 14472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:14.945590 14472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:14.946768 14472 hybrid_clock.cc:648] HybridClock initialized: now 1786115654946768 us; error 56 us; skew 500 ppm
Aug 07 15:14:14 dist-test-slave-3kk6 krb5kdc[14448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115654, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:14.950413 14472 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:14:14.951581 14472 webserver.cc:460] Webserver started at http://127.0.0.1:42043/ using document root <none> and password file <none>
I20260807 15:14:14.952173 14472 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:14.952247 14472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:14.952435 14472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:14.954766 14472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "b292b75b1bda4f559efd4d8b8f498546"
format_stamp: "Formatted at 2026-08-07 15:14:14 on dist-test-slave-3kk6"
server_key: "202f657b12678bfd96b8ffdb73d7632a"
server_key_iv: "4d9b659cc2df2254d3255f3f5ed405fa"
server_key_version: "encryptionkey@0"
I20260807 15:14:14.955410 14472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "b292b75b1bda4f559efd4d8b8f498546"
format_stamp: "Formatted at 2026-08-07 15:14:14 on dist-test-slave-3kk6"
server_key: "202f657b12678bfd96b8ffdb73d7632a"
server_key_iv: "4d9b659cc2df2254d3255f3f5ed405fa"
server_key_version: "encryptionkey@0"
I20260807 15:14:14.959637 14472 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:14:14.962519 14493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:14.964003 14472 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 15:14:14.964159 14472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "b292b75b1bda4f559efd4d8b8f498546"
format_stamp: "Formatted at 2026-08-07 15:14:14 on dist-test-slave-3kk6"
server_key: "202f657b12678bfd96b8ffdb73d7632a"
server_key_iv: "4d9b659cc2df2254d3255f3f5ed405fa"
server_key_version: "encryptionkey@0"
I20260807 15:14:14.964299 14472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:14.969501 14472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:14.973278 14472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:14.973528 14472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:14.981623 14472 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45569
I20260807 15:14:14.981642 14570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45569 every 8 connection(s)
I20260807 15:14:14.982800 14472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:14.985788 14571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:14.991086 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 14472
I20260807 15:14:14.991255 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:14:14.991581 16265 external_mini_cluster.cc:1428] Setting key 0a054f51384da1d7bc92d5f159fd4900
I20260807 15:14:14.992774 14571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b292b75b1bda4f559efd4d8b8f498546: Bootstrap starting.
I20260807 15:14:14.995486 14571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b292b75b1bda4f559efd4d8b8f498546: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:14.996433 14571 log.cc:824] T 00000000000000000000000000000000 P b292b75b1bda4f559efd4d8b8f498546: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:14.997859 14576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:14.993270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:45577 (local address 127.0.0.1:45569)
0807 15:14:14.993791 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:14.993811 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:14.993859 (+ 48us) server_negotiation.cc:407] Connection header received
0807 15:14:14.994562 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:14.994581 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:14.995021 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:14.995391 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:14.996549 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:14.996553 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:14.996566 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:14.996596 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:14.996661 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:14.996688 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:14.996825 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:14.996940 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 15:14:14.997115 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":158,"threads_started":1}
I20260807 15:14:14.998788 14571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b292b75b1bda4f559efd4d8b8f498546: No bootstrap required, opened a new log
I20260807 15:14:15.001025 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:14:15.003944 14571 raft_consensus.cc:348] T 00000000000000000000000000000000 P b292b75b1bda4f559efd4d8b8f498546 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b292b75b1bda4f559efd4d8b8f498546" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45569 } }
I20260807 15:14:15.004330 14571 raft_consensus.cc:374] T 00000000000000000000000000000000 P b292b75b1bda4f559efd4d8b8f498546 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:15.004468 14571 raft_consensus.cc:729] T 00000000000000000000000000000000 P b292b75b1bda4f559efd4d8b8f498546 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b292b75b1bda4f559efd4d8b8f498546, State: Initialized, Role: FOLLOWER
I20260807 15:14:15.005017 14571 consensus_queue.cc:260] T 00000000000000000000000000000000 P b292b75b1bda4f559efd4d8b8f498546 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b292b75b1bda4f559efd4d8b8f498546" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45569 } }
I20260807 15:14:15.005211 14571 raft_consensus.cc:388] T 00000000000000000000000000000000 P b292b75b1bda4f559efd4d8b8f498546 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:15.005293 14571 raft_consensus.cc:482] T 00000000000000000000000000000000 P b292b75b1bda4f559efd4d8b8f498546 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:15.005430 14571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b292b75b1bda4f559efd4d8b8f498546 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:15.005927 14576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:15.003730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:35047 (local address 127.0.0.1:45569)
0807 15:14:15.003923 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:15.003928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:15.004140 (+ 212us) server_negotiation.cc:407] Connection header received
0807 15:14:15.004279 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:15.004281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:15.004345 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:15.004453 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:15.005413 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:15.005418 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:15.005420 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:15.005435 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:15.005470 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:15.005482 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:15.005595 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:15.005752 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 15:14:15.005796 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:14:15.006913 14571 raft_consensus.cc:504] T 00000000000000000000000000000000 P b292b75b1bda4f559efd4d8b8f498546 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b292b75b1bda4f559efd4d8b8f498546" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45569 } }
I20260807 15:14:15.007364 14571 leader_election.cc:302] T 00000000000000000000000000000000 P b292b75b1bda4f559efd4d8b8f498546 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b292b75b1bda4f559efd4d8b8f498546; no voters:
I20260807 15:14:15.007704 14571 leader_election.cc:288] T 00000000000000000000000000000000 P b292b75b1bda4f559efd4d8b8f498546 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:15.007855 14585 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b292b75b1bda4f559efd4d8b8f498546 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:15.008133 14585 raft_consensus.cc:686] T 00000000000000000000000000000000 P b292b75b1bda4f559efd4d8b8f498546 [term 1 LEADER]: Becoming Leader. State: Replica: b292b75b1bda4f559efd4d8b8f498546, State: Running, Role: LEADER
I20260807 15:14:15.009656 14571 sys_catalog.cc:564] T 00000000000000000000000000000000 P b292b75b1bda4f559efd4d8b8f498546 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:15.009701 14585 consensus_queue.cc:237] T 00000000000000000000000000000000 P b292b75b1bda4f559efd4d8b8f498546 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b292b75b1bda4f559efd4d8b8f498546" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45569 } }
I20260807 15:14:15.011969 14587 sys_catalog.cc:455] T 00000000000000000000000000000000 P b292b75b1bda4f559efd4d8b8f498546 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b292b75b1bda4f559efd4d8b8f498546. Latest consensus state: current_term: 1 leader_uuid: "b292b75b1bda4f559efd4d8b8f498546" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b292b75b1bda4f559efd4d8b8f498546" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45569 } } }
I20260807 15:14:15.012471 14587 sys_catalog.cc:458] T 00000000000000000000000000000000 P b292b75b1bda4f559efd4d8b8f498546 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:15.012329 14586 sys_catalog.cc:455] T 00000000000000000000000000000000 P b292b75b1bda4f559efd4d8b8f498546 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b292b75b1bda4f559efd4d8b8f498546" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b292b75b1bda4f559efd4d8b8f498546" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45569 } } }
I20260807 15:14:15.013315 14586 sys_catalog.cc:458] T 00000000000000000000000000000000 P b292b75b1bda4f559efd4d8b8f498546 [sys.catalog]: This master's current role is: LEADER
W20260807 15:14:15.016836 14605 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b292b75b1bda4f559efd4d8b8f498546: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:14:15.016937 14605 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:14:15.017017 14606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:15.017961 14606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:15.025864 14606 catalog_manager.cc:1347] Generated new cluster ID: b162fd2370d64bcbb4a6c5e529bb36a1
I20260807 15:14:15.025964 14606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:15.037210 14606 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:15.037591 14606 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:15.052701 14606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b292b75b1bda4f559efd4d8b8f498546: Generated new TSK 0
I20260807 15:14:15.053565 14606 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:14:15.065956 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 14472
2026-08-07T15:14:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1658 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:15 dist-test-slave-3kk6 krb5kdc[14612](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:15 dist-test-slave-3kk6 krb5kdc[14612](info): set up 2 sockets
Aug 07 15:14:15 dist-test-slave-3kk6 krb5kdc[14612](info): commencing operation
krb5kdc: starting...
W20260807 15:14:16.328714 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.235s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:14:16 dist-test-slave-3kk6 krb5kdc[14612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:14:16.470842 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36857
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:16.593995 14634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:16.594370 14634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:16.594452 14634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:16.598929 14634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:16.599033 14634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:16.599072 14634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:16.599133 14634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:16.599174 14634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:16.604842 14634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36857
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:16.606096 14634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:16.607223 14634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:16.616370 14647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:16.616412 14644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:16.616398 14643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:16.616734 14634 server_base.cc:1034] running on GCE node
I20260807 15:14:16.617307 14634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:16.618484 14634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:16.619652 14634 hybrid_clock.cc:648] HybridClock initialized: now 1786115656619629 us; error 41 us; skew 500 ppm
Aug 07 15:14:16 dist-test-slave-3kk6 krb5kdc[14612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115656, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:16.622581 14634 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:14:16.623605 14634 webserver.cc:460] Webserver started at http://127.0.0.1:33973/ using document root <none> and password file <none>
I20260807 15:14:16.624254 14634 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:16.624322 14634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:16.624581 14634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:16.626752 14634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "911664dd70084c1ead46fff9628367d5"
format_stamp: "Formatted at 2026-08-07 15:14:16 on dist-test-slave-3kk6"
server_key: "aeff5cf613008dd611eb1396f8a5cb00"
server_key_iv: "10119d2d3f414ce0f22bbc1b16259285"
server_key_version: "encryptionkey@0"
I20260807 15:14:16.627355 14634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "911664dd70084c1ead46fff9628367d5"
format_stamp: "Formatted at 2026-08-07 15:14:16 on dist-test-slave-3kk6"
server_key: "aeff5cf613008dd611eb1396f8a5cb00"
server_key_iv: "10119d2d3f414ce0f22bbc1b16259285"
server_key_version: "encryptionkey@0"
I20260807 15:14:16.631238 14634 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:14:16.633733 14658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:16.635087 14634 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:14:16.635236 14634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "911664dd70084c1ead46fff9628367d5"
format_stamp: "Formatted at 2026-08-07 15:14:16 on dist-test-slave-3kk6"
server_key: "aeff5cf613008dd611eb1396f8a5cb00"
server_key_iv: "10119d2d3f414ce0f22bbc1b16259285"
server_key_version: "encryptionkey@0"
I20260807 15:14:16.635358 14634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:16.652029 14634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:16.655297 14634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:16.655529 14634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:16.663446 14634 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36857
I20260807 15:14:16.663475 14730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36857 every 8 connection(s)
I20260807 15:14:16.664512 14634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:16.667546 14731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:16.669747 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 14634
I20260807 15:14:16.669950 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:14:16.670261 16265 external_mini_cluster.cc:1428] Setting key 84d576dc392aa7fc3bc139bcd28fe12a
I20260807 15:14:16.674937 14731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 911664dd70084c1ead46fff9628367d5: Bootstrap starting.
I20260807 15:14:16.677778 14731 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 911664dd70084c1ead46fff9628367d5: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:16.678627 14731 log.cc:824] T 00000000000000000000000000000000 P 911664dd70084c1ead46fff9628367d5: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:16.680732 14731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 911664dd70084c1ead46fff9628367d5: No bootstrap required, opened a new log
I20260807 15:14:16.684383 14731 raft_consensus.cc:348] T 00000000000000000000000000000000 P 911664dd70084c1ead46fff9628367d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "911664dd70084c1ead46fff9628367d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36857 } }
I20260807 15:14:16.684674 14731 raft_consensus.cc:374] T 00000000000000000000000000000000 P 911664dd70084c1ead46fff9628367d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:16.684760 14731 raft_consensus.cc:729] T 00000000000000000000000000000000 P 911664dd70084c1ead46fff9628367d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 911664dd70084c1ead46fff9628367d5, State: Initialized, Role: FOLLOWER
Aug 07 15:14:16 dist-test-slave-3kk6 krb5kdc[14612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 15:14:16.685305 14731 consensus_queue.cc:260] T 00000000000000000000000000000000 P 911664dd70084c1ead46fff9628367d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "911664dd70084c1ead46fff9628367d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36857 } }
I20260807 15:14:16.685489 14731 raft_consensus.cc:388] T 00000000000000000000000000000000 P 911664dd70084c1ead46fff9628367d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:16.685563 14731 raft_consensus.cc:482] T 00000000000000000000000000000000 P 911664dd70084c1ead46fff9628367d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:16.685670 14731 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 911664dd70084c1ead46fff9628367d5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:16.686879 14731 raft_consensus.cc:504] T 00000000000000000000000000000000 P 911664dd70084c1ead46fff9628367d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "911664dd70084c1ead46fff9628367d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36857 } }
I20260807 15:14:16.687297 14731 leader_election.cc:302] T 00000000000000000000000000000000 P 911664dd70084c1ead46fff9628367d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 911664dd70084c1ead46fff9628367d5; no voters:
I20260807 15:14:16.687724 14731 leader_election.cc:288] T 00000000000000000000000000000000 P 911664dd70084c1ead46fff9628367d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:16.687897 14737 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 911664dd70084c1ead46fff9628367d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:16.688122 14737 raft_consensus.cc:686] T 00000000000000000000000000000000 P 911664dd70084c1ead46fff9628367d5 [term 1 LEADER]: Becoming Leader. State: Replica: 911664dd70084c1ead46fff9628367d5, State: Running, Role: LEADER
I20260807 15:14:16.688535 14737 consensus_queue.cc:237] T 00000000000000000000000000000000 P 911664dd70084c1ead46fff9628367d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "911664dd70084c1ead46fff9628367d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36857 } }
I20260807 15:14:16.689113 14731 sys_catalog.cc:564] T 00000000000000000000000000000000 P 911664dd70084c1ead46fff9628367d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:16.690490 14741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 911664dd70084c1ead46fff9628367d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "911664dd70084c1ead46fff9628367d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "911664dd70084c1ead46fff9628367d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36857 } } }
I20260807 15:14:16.690718 14741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 911664dd70084c1ead46fff9628367d5 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:16.692057 14742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 911664dd70084c1ead46fff9628367d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 911664dd70084c1ead46fff9628367d5. Latest consensus state: current_term: 1 leader_uuid: "911664dd70084c1ead46fff9628367d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "911664dd70084c1ead46fff9628367d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36857 } } }
I20260807 15:14:16.692174 14742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 911664dd70084c1ead46fff9628367d5 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:16.692725 14734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:16.671951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:32911 (local address 127.0.0.1:36857)
0807 15:14:16.672460 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:16.672483 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:16.672534 (+ 51us) server_negotiation.cc:407] Connection header received
0807 15:14:16.673627 (+ 1093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:16.673654 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:16.674127 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:16.674536 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:16.675481 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:16.677000 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:16.678050 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:16.678575 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:16.678802 (+ 227us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:16.685982 (+ 7180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:16.685987 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:16.686001 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:16.686034 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:16.688489 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:16.689186 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:16.689205 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:16.689212 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:16.689362 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:16.689792 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:16.689796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:16.689798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:16.690361 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:16.690594 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:16.691454 (+ 860us) server_negotiation.cc:299] Negotiation successful
0807 15:14:16.691716 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":144,"threads_started":1}
W20260807 15:14:16.694592 14756 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 911664dd70084c1ead46fff9628367d5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:14:16.694716 14756 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:14:16.694845 14751 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:16.695832 14751 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:16.696319 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:14:16.702486 14751 catalog_manager.cc:1347] Generated new cluster ID: b92e1ab9bc7c4f569c6bf12fbe206834
I20260807 15:14:16.702565 14751 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:16.708472 14734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:16.698745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:49889 (local address 127.0.0.1:36857)
0807 15:14:16.698865 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:16.698869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:16.699108 (+ 239us) server_negotiation.cc:407] Connection header received
0807 15:14:16.699281 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:16.699285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:16.699339 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:16.699424 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:16.700140 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:16.700852 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:16.702093 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:16.702505 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:16.702590 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:16.704650 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:16.704654 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:16.704657 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:16.704683 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:16.706416 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:16.707035 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:16.707038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:16.707040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:16.707129 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:16.707506 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:16.707509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:16.707510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:16.707724 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:16.707875 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:16.708185 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 15:14:16.708288 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:14:16.710645 14751 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:16.711892 14751 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:16.724462 14751 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 911664dd70084c1ead46fff9628367d5: Generated new TSK 0
I20260807 15:14:16.725427 14751 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:14:16.729234 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 14634
W20260807 15:14:16.748574 14610 connection.cc:535] client connection to 127.0.0.1:36857 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36857): Connection reset by peer (error 104)
2026-08-07T15:14:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1669 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:16 dist-test-slave-3kk6 krb5kdc[14773](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:16 dist-test-slave-3kk6 krb5kdc[14773](info): set up 2 sockets
Aug 07 15:14:16 dist-test-slave-3kk6 krb5kdc[14773](info): commencing operation
krb5kdc: starting...
W20260807 15:14:17.974889 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:14:18 dist-test-slave-3kk6 krb5kdc[14773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115658, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:14:18.126330 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36363
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:18.247592 14792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:18.247906 14792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:18.247951 14792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:18.251961 14792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:18.252029 14792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:18.252049 14792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:18.252066 14792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:18.252081 14792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:18.257563 14792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36363
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:18.258692 14792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:18.259702 14792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:18.268152 14804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:18.268239 14810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:18.268278 14808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:18.268618 14792 server_base.cc:1034] running on GCE node
I20260807 15:14:18.269174 14792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:18.270303 14792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:18.271494 14792 hybrid_clock.cc:648] HybridClock initialized: now 1786115658271445 us; error 61 us; skew 500 ppm
Aug 07 15:14:18 dist-test-slave-3kk6 krb5kdc[14773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115658, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:18.274490 14792 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:14:18.275548 14792 webserver.cc:460] Webserver started at http://127.0.0.1:36095/ using document root <none> and password file <none>
I20260807 15:14:18.276136 14792 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:18.276232 14792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:18.276477 14792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:18.278586 14792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "0a547236aa5540ef96622d67985b0b5f"
format_stamp: "Formatted at 2026-08-07 15:14:18 on dist-test-slave-3kk6"
server_key: "3230c991e810cce61b4e4384d56ad8a3"
server_key_iv: "6b849590ff1b744783c98386e4d6b043"
server_key_version: "encryptionkey@0"
I20260807 15:14:18.279270 14792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "0a547236aa5540ef96622d67985b0b5f"
format_stamp: "Formatted at 2026-08-07 15:14:18 on dist-test-slave-3kk6"
server_key: "3230c991e810cce61b4e4384d56ad8a3"
server_key_iv: "6b849590ff1b744783c98386e4d6b043"
server_key_version: "encryptionkey@0"
I20260807 15:14:18.283365 14792 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 15:14:18.286065 14820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:18.287482 14792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:14:18.287633 14792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "0a547236aa5540ef96622d67985b0b5f"
format_stamp: "Formatted at 2026-08-07 15:14:18 on dist-test-slave-3kk6"
server_key: "3230c991e810cce61b4e4384d56ad8a3"
server_key_iv: "6b849590ff1b744783c98386e4d6b043"
server_key_version: "encryptionkey@0"
I20260807 15:14:18.287777 14792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:18.304154 14792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:18.307516 14792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:18.307749 14792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:18.315452 14792 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36363
I20260807 15:14:18.315542 14891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36363 every 8 connection(s)
I20260807 15:14:18.316457 14792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:18.319530 14892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:18.325120 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 14792
I20260807 15:14:18.325302 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:14:18.325634 16265 external_mini_cluster.cc:1428] Setting key 181ae3bbc23ae6cc316469aeff40f289
I20260807 15:14:18.326053 14892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a547236aa5540ef96622d67985b0b5f: Bootstrap starting.
I20260807 15:14:18.328934 14892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a547236aa5540ef96622d67985b0b5f: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:18.329931 14892 log.cc:824] T 00000000000000000000000000000000 P 0a547236aa5540ef96622d67985b0b5f: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:18.332489 14892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a547236aa5540ef96622d67985b0b5f: No bootstrap required, opened a new log
I20260807 15:14:18.335726 14895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:18.327260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:50357 (local address 127.0.0.1:36363)
0807 15:14:18.327783 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:18.327799 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:18.327838 (+ 39us) server_negotiation.cc:407] Connection header received
0807 15:14:18.328496 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:18.328514 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:18.328861 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:18.329219 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:18.329627 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:18.330921 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:18.332282 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:18.332806 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:18.332940 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:18.333951 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:18.333957 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:18.333969 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:18.334362 (+ 393us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:18.334429 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:18.334452 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:18.334557 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:18.334804 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 15:14:18.334963 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":165,"threads_started":1}
I20260807 15:14:18.337095 14892 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a547236aa5540ef96622d67985b0b5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a547236aa5540ef96622d67985b0b5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36363 } }
I20260807 15:14:18.337383 14892 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a547236aa5540ef96622d67985b0b5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:18.337471 14892 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a547236aa5540ef96622d67985b0b5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a547236aa5540ef96622d67985b0b5f, State: Initialized, Role: FOLLOWER
I20260807 15:14:18.337990 14892 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a547236aa5540ef96622d67985b0b5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a547236aa5540ef96622d67985b0b5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36363 } }
I20260807 15:14:18.338181 14892 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a547236aa5540ef96622d67985b0b5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:18.338333 14892 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a547236aa5540ef96622d67985b0b5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:18.338455 14892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a547236aa5540ef96622d67985b0b5f [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:18.338862 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:14:18.339658 14892 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a547236aa5540ef96622d67985b0b5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a547236aa5540ef96622d67985b0b5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36363 } }
I20260807 15:14:18.340042 14892 leader_election.cc:302] T 00000000000000000000000000000000 P 0a547236aa5540ef96622d67985b0b5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a547236aa5540ef96622d67985b0b5f; no voters:
I20260807 15:14:18.340467 14892 leader_election.cc:288] T 00000000000000000000000000000000 P 0a547236aa5540ef96622d67985b0b5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:18.340530 14903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a547236aa5540ef96622d67985b0b5f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:18.340806 14903 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a547236aa5540ef96622d67985b0b5f [term 1 LEADER]: Becoming Leader. State: Replica: 0a547236aa5540ef96622d67985b0b5f, State: Running, Role: LEADER
I20260807 15:14:18.341202 14903 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a547236aa5540ef96622d67985b0b5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a547236aa5540ef96622d67985b0b5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36363 } }
I20260807 15:14:18.341960 14892 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a547236aa5540ef96622d67985b0b5f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:18.343551 14905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a547236aa5540ef96622d67985b0b5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a547236aa5540ef96622d67985b0b5f. Latest consensus state: current_term: 1 leader_uuid: "0a547236aa5540ef96622d67985b0b5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a547236aa5540ef96622d67985b0b5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36363 } } }
I20260807 15:14:18.343698 14905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a547236aa5540ef96622d67985b0b5f [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:18.344369 14904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a547236aa5540ef96622d67985b0b5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a547236aa5540ef96622d67985b0b5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a547236aa5540ef96622d67985b0b5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36363 } } }
I20260807 15:14:18.344501 14904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a547236aa5540ef96622d67985b0b5f [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:18.346352 14895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:18.341099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:35589 (local address 127.0.0.1:36363)
0807 15:14:18.341189 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:18.341194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:18.341652 (+ 458us) server_negotiation.cc:407] Connection header received
0807 15:14:18.341777 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:18.341780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:18.341843 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:18.341909 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:18.342323 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:18.343184 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:18.344335 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:18.344647 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:18.344756 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:18.345878 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:18.345882 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:18.345885 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:18.345905 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:18.345938 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:18.345949 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:18.346051 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:18.346176 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 15:14:18.346209 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 15:14:18.348116 14921 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a547236aa5540ef96622d67985b0b5f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:14:18.348220 14921 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:14:18.348320 14913 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:18.349113 14913 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:18.355904 14913 catalog_manager.cc:1347] Generated new cluster ID: 6d76d813f09640c18f703f86b92a0450
I20260807 15:14:18.355978 14913 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:18.384639 14913 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:18.385810 14913 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:18.392472 14913 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a547236aa5540ef96622d67985b0b5f: Generated new TSK 0
I20260807 15:14:18.393215 14913 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:14:18.402323 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 14792
W20260807 15:14:18.413264 14771 connection.cc:535] client connection to 127.0.0.1:36363 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36363): Connection reset by peer (error 104)
2026-08-07T15:14:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1664 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:18 dist-test-slave-3kk6 krb5kdc[14930](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:18 dist-test-slave-3kk6 krb5kdc[14930](info): set up 2 sockets
Aug 07 15:14:18 dist-test-slave-3kk6 krb5kdc[14930](info): commencing operation
krb5kdc: starting...
W20260807 15:14:19.644810 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:14:19 dist-test-slave-3kk6 krb5kdc[14930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 15:14:19.790524 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39667
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:19.913951 14947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:19.914301 14947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:19.914350 14947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:19.918713 14947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:19.918809 14947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:19.918831 14947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:19.918848 14947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:19.918864 14947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:19.924813 14947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39667
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:19.926123 14947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:19.927349 14947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:19.937278 14958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:19.937345 14955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:19.937323 14963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:19.937544 14947 server_base.cc:1034] running on GCE node
I20260807 15:14:19.938349 14947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:19.939630 14947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:19.940862 14947 hybrid_clock.cc:648] HybridClock initialized: now 1786115659940833 us; error 47 us; skew 500 ppm
Aug 07 15:14:19 dist-test-slave-3kk6 krb5kdc[14930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115659, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:19.944185 14947 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 15:14:19.945322 14947 webserver.cc:460] Webserver started at http://127.0.0.1:42723/ using document root <none> and password file <none>
I20260807 15:14:19.945953 14947 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:19.946055 14947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:19.946314 14947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:19.948719 14947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "ab8898154a834ed793f083b9ba3e19bd"
format_stamp: "Formatted at 2026-08-07 15:14:19 on dist-test-slave-3kk6"
server_key: "3c62645c6d09152f0ee49ba275d91de4"
server_key_iv: "2fddf6fa45c572172dc8c5696b4635ad"
server_key_version: "encryptionkey@0"
I20260807 15:14:19.949417 14947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "ab8898154a834ed793f083b9ba3e19bd"
format_stamp: "Formatted at 2026-08-07 15:14:19 on dist-test-slave-3kk6"
server_key: "3c62645c6d09152f0ee49ba275d91de4"
server_key_iv: "2fddf6fa45c572172dc8c5696b4635ad"
server_key_version: "encryptionkey@0"
I20260807 15:14:19.953826 14947 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 15:14:19.956827 14970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:19.958220 14947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 15:14:19.958400 14947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "ab8898154a834ed793f083b9ba3e19bd"
format_stamp: "Formatted at 2026-08-07 15:14:19 on dist-test-slave-3kk6"
server_key: "3c62645c6d09152f0ee49ba275d91de4"
server_key_iv: "2fddf6fa45c572172dc8c5696b4635ad"
server_key_version: "encryptionkey@0"
I20260807 15:14:19.958545 14947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:19.964095 14947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:19.968129 14947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:19.968389 14947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:19.977099 14947 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39667
I20260807 15:14:19.977124 15041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39667 every 8 connection(s)
I20260807 15:14:19.978206 14947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:19.979877 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 14947
I20260807 15:14:19.980050 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:14:19.980424 16265 external_mini_cluster.cc:1428] Setting key 16484e7647233f0524ceb1885ff337ce
I20260807 15:14:19.982141 15043 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:19.986815 15048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:19.982171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:45127 (local address 127.0.0.1:39667)
0807 15:14:19.982764 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:19.982785 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:19.982836 (+ 51us) server_negotiation.cc:407] Connection header received
0807 15:14:19.983633 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:19.983655 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:19.983967 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:19.984283 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:19.985327 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:19.985332 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:19.985346 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:19.985358 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:19.985420 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:19.985440 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:19.985562 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:19.985707 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 15:14:19.985901 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":170,"threads_started":1}
I20260807 15:14:19.989761 15043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab8898154a834ed793f083b9ba3e19bd: Bootstrap starting.
I20260807 15:14:19.990077 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:14:19.992683 15043 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab8898154a834ed793f083b9ba3e19bd: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:19.993891 15043 log.cc:824] T 00000000000000000000000000000000 P ab8898154a834ed793f083b9ba3e19bd: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:19.994850 15048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:19.992486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.97:46377 (local address 127.0.0.1:39667)
0807 15:14:19.992646 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:19.992651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:19.992870 (+ 219us) server_negotiation.cc:407] Connection header received
0807 15:14:19.993101 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:19.993104 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:19.993142 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:19.993213 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:19.994200 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:19.994203 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:19.994205 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:19.994218 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:19.994249 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:19.994259 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:19.994342 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:19.994610 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 15:14:19.994694 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 15:14:19.996737 15043 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab8898154a834ed793f083b9ba3e19bd: No bootstrap required, opened a new log
I20260807 15:14:20.001161 15043 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab8898154a834ed793f083b9ba3e19bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab8898154a834ed793f083b9ba3e19bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39667 } }
I20260807 15:14:20.001456 15043 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab8898154a834ed793f083b9ba3e19bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:20.001494 15043 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab8898154a834ed793f083b9ba3e19bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab8898154a834ed793f083b9ba3e19bd, State: Initialized, Role: FOLLOWER
I20260807 15:14:20.001895 15043 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab8898154a834ed793f083b9ba3e19bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab8898154a834ed793f083b9ba3e19bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39667 } }
I20260807 15:14:20.002015 15043 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab8898154a834ed793f083b9ba3e19bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:20.002059 15043 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab8898154a834ed793f083b9ba3e19bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:20.002135 15043 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab8898154a834ed793f083b9ba3e19bd [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:20.003324 15043 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab8898154a834ed793f083b9ba3e19bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab8898154a834ed793f083b9ba3e19bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39667 } }
I20260807 15:14:20.003671 15043 leader_election.cc:302] T 00000000000000000000000000000000 P ab8898154a834ed793f083b9ba3e19bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab8898154a834ed793f083b9ba3e19bd; no voters:
I20260807 15:14:20.004011 15043 leader_election.cc:288] T 00000000000000000000000000000000 P ab8898154a834ed793f083b9ba3e19bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:20.004215 15058 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab8898154a834ed793f083b9ba3e19bd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:20.004558 15058 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab8898154a834ed793f083b9ba3e19bd [term 1 LEADER]: Becoming Leader. State: Replica: ab8898154a834ed793f083b9ba3e19bd, State: Running, Role: LEADER
I20260807 15:14:20.004914 15058 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab8898154a834ed793f083b9ba3e19bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab8898154a834ed793f083b9ba3e19bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39667 } }
I20260807 15:14:20.005505 15043 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab8898154a834ed793f083b9ba3e19bd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:20.006853 15060 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab8898154a834ed793f083b9ba3e19bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab8898154a834ed793f083b9ba3e19bd. Latest consensus state: current_term: 1 leader_uuid: "ab8898154a834ed793f083b9ba3e19bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab8898154a834ed793f083b9ba3e19bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39667 } } }
I20260807 15:14:20.006995 15060 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab8898154a834ed793f083b9ba3e19bd [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:20.007331 15059 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab8898154a834ed793f083b9ba3e19bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab8898154a834ed793f083b9ba3e19bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab8898154a834ed793f083b9ba3e19bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39667 } } }
I20260807 15:14:20.007431 15059 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab8898154a834ed793f083b9ba3e19bd [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:20.007761 15068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:20.010847 15068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:20.017102 15068 catalog_manager.cc:1347] Generated new cluster ID: f5b1dcfa4e2145ff8c8e70e4b977dbed
I20260807 15:14:20.017203 15068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:20.027822 15068 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:20.028268 15068 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:20.036437 15068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab8898154a834ed793f083b9ba3e19bd: Generated new TSK 0
I20260807 15:14:20.037436 15068 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:14:20.042395 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 14947
2026-08-07T15:14:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1640 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:20 dist-test-slave-3kk6 krb5kdc[15088](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:20 dist-test-slave-3kk6 krb5kdc[15088](info): set up 2 sockets
Aug 07 15:14:20 dist-test-slave-3kk6 krb5kdc[15088](info): commencing operation
krb5kdc: starting...
W20260807 15:14:21.310628 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:14:21 dist-test-slave-3kk6 krb5kdc[15088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:14:21.460399 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:45365
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:43271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:45365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:21.581336 15109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:21.581956 15109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:21.582085 15109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:21.586489 15109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:21.586627 15109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:21.586691 15109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:21.586740 15109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:21.586784 15109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:21.592676 15109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:43271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:45365
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:45365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:21.594033 15109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:21.595228 15109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:21.605403 15117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:21.605427 15120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:21.605573 15109 server_base.cc:1034] running on GCE node
W20260807 15:14:21.605403 15118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:21.606504 15109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:21.607849 15109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:21.609036 15109 hybrid_clock.cc:648] HybridClock initialized: now 1786115661609022 us; error 48 us; skew 500 ppm
Aug 07 15:14:21 dist-test-slave-3kk6 krb5kdc[15088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115661, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:21.612483 15109 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:14:21.613646 15109 webserver.cc:460] Webserver started at http://127.15.226.126:41115/ using document root <none> and password file <none>
I20260807 15:14:21.614284 15109 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:21.614391 15109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:21.614710 15109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:21.617074 15109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "ff7e703994714fc1bf4bfe860afe46ec"
format_stamp: "Formatted at 2026-08-07 15:14:21 on dist-test-slave-3kk6"
server_key: "cf6cbc6231ebb71b554ed1e827501752"
server_key_iv: "c078621966a0c5edd915fccb17b6956b"
server_key_version: "encryptionkey@0"
I20260807 15:14:21.617733 15109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "ff7e703994714fc1bf4bfe860afe46ec"
format_stamp: "Formatted at 2026-08-07 15:14:21 on dist-test-slave-3kk6"
server_key: "cf6cbc6231ebb71b554ed1e827501752"
server_key_iv: "c078621966a0c5edd915fccb17b6956b"
server_key_version: "encryptionkey@0"
I20260807 15:14:21.622092 15109 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260807 15:14:21.625144 15128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:21.626504 15109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 15:14:21.626717 15109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "ff7e703994714fc1bf4bfe860afe46ec"
format_stamp: "Formatted at 2026-08-07 15:14:21 on dist-test-slave-3kk6"
server_key: "cf6cbc6231ebb71b554ed1e827501752"
server_key_iv: "c078621966a0c5edd915fccb17b6956b"
server_key_version: "encryptionkey@0"
I20260807 15:14:21.626899 15109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:21.654897 15109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:21.658929 15109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:21.659183 15109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:21.667583 15109 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:45365
I20260807 15:14:21.667641 15195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:45365 every 8 connection(s)
I20260807 15:14:21.668733 15109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:21.669418 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 15109
I20260807 15:14:21.669608 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:14:21.669986 16265 external_mini_cluster.cc:1428] Setting key e54696481bc19d317f64fbc20d7a3d78
I20260807 15:14:21.673072 15196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:21.679112 15196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff7e703994714fc1bf4bfe860afe46ec: Bootstrap starting.
Aug 07 15:14:21 dist-test-slave-3kk6 krb5kdc[15088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:14:21.682185 15196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff7e703994714fc1bf4bfe860afe46ec: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:21.683207 15196 log.cc:824] T 00000000000000000000000000000000 P ff7e703994714fc1bf4bfe860afe46ec: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:21.685770 15196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff7e703994714fc1bf4bfe860afe46ec: No bootstrap required, opened a new log
I20260807 15:14:21.685961 15200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:21.671480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55181 (local address 127.15.226.126:45365)
0807 15:14:21.671952 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:21.671969 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:21.672009 (+ 40us) server_negotiation.cc:407] Connection header received
0807 15:14:21.672723 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:21.672741 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:21.673108 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:21.673527 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:21.674444 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:21.675537 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:21.676456 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:21.676945 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:21.677120 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:21.680095 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:21.680103 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:21.680121 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:21.680157 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:21.682389 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:21.683059 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:21.683069 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:21.683078 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:21.683230 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:21.683666 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:21.683670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:21.683672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:21.684157 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:21.684399 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:21.684727 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 15:14:21.684979 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":143,"threads_started":1}
I20260807 15:14:21.689347 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:14:21.690279 15196 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff7e703994714fc1bf4bfe860afe46ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff7e703994714fc1bf4bfe860afe46ec" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45365 } }
I20260807 15:14:21.690579 15196 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff7e703994714fc1bf4bfe860afe46ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:21.690640 15196 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff7e703994714fc1bf4bfe860afe46ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff7e703994714fc1bf4bfe860afe46ec, State: Initialized, Role: FOLLOWER
I20260807 15:14:21.691200 15196 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff7e703994714fc1bf4bfe860afe46ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff7e703994714fc1bf4bfe860afe46ec" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45365 } }
I20260807 15:14:21.691381 15196 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff7e703994714fc1bf4bfe860afe46ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:21.691483 15196 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff7e703994714fc1bf4bfe860afe46ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:21.691620 15196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff7e703994714fc1bf4bfe860afe46ec [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:21.692940 15196 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff7e703994714fc1bf4bfe860afe46ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff7e703994714fc1bf4bfe860afe46ec" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45365 } }
I20260807 15:14:21.693399 15196 leader_election.cc:302] T 00000000000000000000000000000000 P ff7e703994714fc1bf4bfe860afe46ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff7e703994714fc1bf4bfe860afe46ec; no voters:
I20260807 15:14:21.693842 15196 leader_election.cc:288] T 00000000000000000000000000000000 P ff7e703994714fc1bf4bfe860afe46ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:21.694072 15210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff7e703994714fc1bf4bfe860afe46ec [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:21.694396 15210 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff7e703994714fc1bf4bfe860afe46ec [term 1 LEADER]: Becoming Leader. State: Replica: ff7e703994714fc1bf4bfe860afe46ec, State: Running, Role: LEADER
I20260807 15:14:21.694823 15210 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff7e703994714fc1bf4bfe860afe46ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff7e703994714fc1bf4bfe860afe46ec" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45365 } }
I20260807 15:14:21.695601 15196 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff7e703994714fc1bf4bfe860afe46ec [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:21.696796 15212 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff7e703994714fc1bf4bfe860afe46ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff7e703994714fc1bf4bfe860afe46ec. Latest consensus state: current_term: 1 leader_uuid: "ff7e703994714fc1bf4bfe860afe46ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff7e703994714fc1bf4bfe860afe46ec" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45365 } } }
I20260807 15:14:21.696981 15212 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff7e703994714fc1bf4bfe860afe46ec [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:21.697304 15211 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff7e703994714fc1bf4bfe860afe46ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff7e703994714fc1bf4bfe860afe46ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff7e703994714fc1bf4bfe860afe46ec" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45365 } } }
I20260807 15:14:21.697402 15223 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:21.697415 15211 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff7e703994714fc1bf4bfe860afe46ec [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:21.701026 15223 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:21.708269 15223 catalog_manager.cc:1347] Generated new cluster ID: 10afd9cbf1d44ad986cb7fdfbe9c3dec
I20260807 15:14:21.708361 15223 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:21.709030 15200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:21.691660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49149 (local address 127.15.226.126:45365)
0807 15:14:21.691831 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:21.691836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:21.692090 (+ 254us) server_negotiation.cc:407] Connection header received
0807 15:14:21.692179 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:21.692184 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:21.692241 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:21.692342 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:21.693049 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:21.693824 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:21.695216 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:21.695563 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:21.695651 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:21.702262 (+ 6611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:21.702268 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:21.702271 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:21.702312 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:21.706102 (+ 3790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:21.707017 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:21.707022 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:21.707024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:21.707137 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:21.707591 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:21.707595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:21.707597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:21.707842 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:21.707995 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:21.708749 (+ 754us) server_negotiation.cc:299] Negotiation successful
0807 15:14:21.708864 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 15:14:21.738237 15223 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:21.739683 15223 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:21.755842 15223 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff7e703994714fc1bf4bfe860afe46ec: Generated new TSK 0
I20260807 15:14:21.756879 15223 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:14:21.763192 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 15109
W20260807 15:14:21.780915 15085 connection.cc:535] client connection to 127.15.226.126:45365 recv error: Network error: failed to read from TLS socket (remote: 127.15.226.126:45365): Connection reset by peer (error 104)
2026-08-07T15:14:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1727 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:21 dist-test-slave-3kk6 krb5kdc[15238](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:21 dist-test-slave-3kk6 krb5kdc[15238](info): set up 2 sockets
Aug 07 15:14:21 dist-test-slave-3kk6 krb5kdc[15238](info): commencing operation
krb5kdc: starting...
W20260807 15:14:23.011430 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:14:23 dist-test-slave-3kk6 krb5kdc[15238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:14:23.154117 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:40805
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:45015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:40805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:23.270759 15254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:23.271106 15254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:23.271188 15254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:23.275290 15254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:23.275416 15254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:23.275456 15254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:23.275496 15254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:23.275537 15254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:23.280969 15254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:45015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:40805
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:40805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:23.282140 15254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:23.283252 15254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:23.292089 15261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:23.292289 15254 server_base.cc:1034] running on GCE node
W20260807 15:14:23.292057 15260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:23.292066 15264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:23.293020 15254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:23.294103 15254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:23.295269 15254 hybrid_clock.cc:648] HybridClock initialized: now 1786115663295258 us; error 36 us; skew 500 ppm
Aug 07 15:14:23 dist-test-slave-3kk6 krb5kdc[15238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115663, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:23.298616 15254 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:14:23.299683 15254 webserver.cc:460] Webserver started at http://127.15.226.126:37553/ using document root <none> and password file <none>
I20260807 15:14:23.300225 15254 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:23.300285 15254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:23.300472 15254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:23.302598 15254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "415d21423f8e4548893f251a02c16648"
format_stamp: "Formatted at 2026-08-07 15:14:23 on dist-test-slave-3kk6"
server_key: "69829cbd94e3a6dbff09a61d7889ff11"
server_key_iv: "86698438ec13089a7aaebbe8d78d21dd"
server_key_version: "encryptionkey@0"
I20260807 15:14:23.303239 15254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "415d21423f8e4548893f251a02c16648"
format_stamp: "Formatted at 2026-08-07 15:14:23 on dist-test-slave-3kk6"
server_key: "69829cbd94e3a6dbff09a61d7889ff11"
server_key_iv: "86698438ec13089a7aaebbe8d78d21dd"
server_key_version: "encryptionkey@0"
I20260807 15:14:23.307150 15254 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 15:14:23.309645 15272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:23.310997 15254 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:14:23.311156 15254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "415d21423f8e4548893f251a02c16648"
format_stamp: "Formatted at 2026-08-07 15:14:23 on dist-test-slave-3kk6"
server_key: "69829cbd94e3a6dbff09a61d7889ff11"
server_key_iv: "86698438ec13089a7aaebbe8d78d21dd"
server_key_version: "encryptionkey@0"
I20260807 15:14:23.311280 15254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:23.344316 15254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:23.347957 15254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:23.348189 15254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:23.356252 15254 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:40805
I20260807 15:14:23.356281 15348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:40805 every 8 connection(s)
I20260807 15:14:23.357293 15254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:23.360167 15349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:23.362962 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 15254
I20260807 15:14:23.363121 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:14:23.363440 16265 external_mini_cluster.cc:1428] Setting key 43a8b697bec98cf1d5238c3752a3d53b
I20260807 15:14:23.366837 15349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 415d21423f8e4548893f251a02c16648: Bootstrap starting.
I20260807 15:14:23.369572 15349 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 415d21423f8e4548893f251a02c16648: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:23.370448 15349 log.cc:824] T 00000000000000000000000000000000 P 415d21423f8e4548893f251a02c16648: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:23.372614 15349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 415d21423f8e4548893f251a02c16648: No bootstrap required, opened a new log
I20260807 15:14:23.373904 15352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:23.364995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42793 (local address 127.15.226.126:40805)
0807 15:14:23.365468 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:23.365492 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:23.365539 (+ 47us) server_negotiation.cc:407] Connection header received
0807 15:14:23.366370 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:23.366392 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:23.366844 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:23.367211 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:23.367639 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:23.369172 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:23.370124 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:23.370952 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:23.371137 (+ 185us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:23.372161 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:23.372165 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:23.372178 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:23.372570 (+ 392us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:23.372625 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:23.372647 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:23.372801 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:23.373001 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 15:14:23.373162 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":122,"threads_started":1}
I20260807 15:14:23.376768 15349 raft_consensus.cc:348] T 00000000000000000000000000000000 P 415d21423f8e4548893f251a02c16648 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415d21423f8e4548893f251a02c16648" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 40805 } }
I20260807 15:14:23.377063 15349 raft_consensus.cc:374] T 00000000000000000000000000000000 P 415d21423f8e4548893f251a02c16648 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:23.377147 15349 raft_consensus.cc:729] T 00000000000000000000000000000000 P 415d21423f8e4548893f251a02c16648 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 415d21423f8e4548893f251a02c16648, State: Initialized, Role: FOLLOWER
I20260807 15:14:23.377156 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:14:23.377619 15349 consensus_queue.cc:260] T 00000000000000000000000000000000 P 415d21423f8e4548893f251a02c16648 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415d21423f8e4548893f251a02c16648" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 40805 } }
I20260807 15:14:23.377806 15349 raft_consensus.cc:388] T 00000000000000000000000000000000 P 415d21423f8e4548893f251a02c16648 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:23.377873 15349 raft_consensus.cc:482] T 00000000000000000000000000000000 P 415d21423f8e4548893f251a02c16648 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:23.377976 15349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 415d21423f8e4548893f251a02c16648 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:23.379281 15349 raft_consensus.cc:504] T 00000000000000000000000000000000 P 415d21423f8e4548893f251a02c16648 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415d21423f8e4548893f251a02c16648" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 40805 } }
I20260807 15:14:23.379740 15349 leader_election.cc:302] T 00000000000000000000000000000000 P 415d21423f8e4548893f251a02c16648 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 415d21423f8e4548893f251a02c16648; no voters:
I20260807 15:14:23.380108 15349 leader_election.cc:288] T 00000000000000000000000000000000 P 415d21423f8e4548893f251a02c16648 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:23.380204 15363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 415d21423f8e4548893f251a02c16648 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:23.380465 15363 raft_consensus.cc:686] T 00000000000000000000000000000000 P 415d21423f8e4548893f251a02c16648 [term 1 LEADER]: Becoming Leader. State: Replica: 415d21423f8e4548893f251a02c16648, State: Running, Role: LEADER
I20260807 15:14:23.380863 15363 consensus_queue.cc:237] T 00000000000000000000000000000000 P 415d21423f8e4548893f251a02c16648 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415d21423f8e4548893f251a02c16648" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 40805 } }
I20260807 15:14:23.381455 15349 sys_catalog.cc:564] T 00000000000000000000000000000000 P 415d21423f8e4548893f251a02c16648 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:23.382443 15364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 415d21423f8e4548893f251a02c16648 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "415d21423f8e4548893f251a02c16648" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415d21423f8e4548893f251a02c16648" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 40805 } } }
I20260807 15:14:23.382617 15364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 415d21423f8e4548893f251a02c16648 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:23.382917 15364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 415d21423f8e4548893f251a02c16648 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 415d21423f8e4548893f251a02c16648. Latest consensus state: current_term: 1 leader_uuid: "415d21423f8e4548893f251a02c16648" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "415d21423f8e4548893f251a02c16648" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 40805 } } }
I20260807 15:14:23.383059 15364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 415d21423f8e4548893f251a02c16648 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:23.384508 15375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:23.385131 15352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:23.379507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34877 (local address 127.15.226.126:40805)
0807 15:14:23.379636 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:23.379640 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:23.379923 (+ 283us) server_negotiation.cc:407] Connection header received
0807 15:14:23.380099 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:23.380103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:23.380156 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:23.380242 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:23.380700 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:23.381450 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:23.382847 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:23.383190 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:23.383274 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:23.384402 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:23.384406 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:23.384409 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:23.384425 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:23.384452 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:23.384465 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:23.384571 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:23.384955 (+ 384us) server_negotiation.cc:299] Negotiation successful
0807 15:14:23.384998 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 15:14:23.387856 15375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:23.393544 15375 catalog_manager.cc:1347] Generated new cluster ID: 2bd9a5eebd584c8f91fa42dfeb00838d
I20260807 15:14:23.393622 15375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:23.402235 15375 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:23.403458 15375 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:23.408878 15375 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 415d21423f8e4548893f251a02c16648: Generated new TSK 0
I20260807 15:14:23.409622 15375 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:14:23.415287 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 15254
W20260807 15:14:23.425824 15236 connection.cc:535] client connection to 127.15.226.126:40805 recv error: Network error: failed to read from TLS socket (remote: 127.15.226.126:40805): Connection reset by peer (error 104)
2026-08-07T15:14:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1644 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:23 dist-test-slave-3kk6 krb5kdc[15398](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:23 dist-test-slave-3kk6 krb5kdc[15398](info): set up 2 sockets
Aug 07 15:14:23 dist-test-slave-3kk6 krb5kdc[15398](info): commencing operation
krb5kdc: starting...
W20260807 15:14:24.686056 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:14:24 dist-test-slave-3kk6 krb5kdc[15398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:14:24.824090 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:36329
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:35769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:36329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:24.942130 15416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:24.942503 15416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:24.942603 15416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:24.946823 15416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:24.946988 15416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:24.947033 15416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:24.947077 15416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:24.947114 15416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:24.953025 15416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:35769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:36329
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:36329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:24.954372 15416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:24.955582 15416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 15:14:24.965149 15416 server_base.cc:1034] running on GCE node
W20260807 15:14:24.965091 15422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:24.965109 15426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:24.965091 15423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:24.966084 15416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:24.967430 15416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:24.968616 15416 hybrid_clock.cc:648] HybridClock initialized: now 1786115664968592 us; error 56 us; skew 500 ppm
Aug 07 15:14:24 dist-test-slave-3kk6 krb5kdc[15398](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115664, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:24.972081 15416 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:14:24.973116 15416 webserver.cc:460] Webserver started at http://127.15.226.126:42813/ using document root <none> and password file <none>
I20260807 15:14:24.973727 15416 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:24.973802 15416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:24.974059 15416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:24.976441 15416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "708d2dd2b93e420781b2bc2d1f9dc68e"
format_stamp: "Formatted at 2026-08-07 15:14:24 on dist-test-slave-3kk6"
server_key: "a924f0393d50debac607367555d72d3e"
server_key_iv: "9b1347dc6c01792ccc3bf6a55a84e38f"
server_key_version: "encryptionkey@0"
I20260807 15:14:24.977136 15416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "708d2dd2b93e420781b2bc2d1f9dc68e"
format_stamp: "Formatted at 2026-08-07 15:14:24 on dist-test-slave-3kk6"
server_key: "a924f0393d50debac607367555d72d3e"
server_key_iv: "9b1347dc6c01792ccc3bf6a55a84e38f"
server_key_version: "encryptionkey@0"
I20260807 15:14:24.981428 15416 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260807 15:14:24.984288 15436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:24.985805 15416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:14:24.985994 15416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "708d2dd2b93e420781b2bc2d1f9dc68e"
format_stamp: "Formatted at 2026-08-07 15:14:24 on dist-test-slave-3kk6"
server_key: "a924f0393d50debac607367555d72d3e"
server_key_iv: "9b1347dc6c01792ccc3bf6a55a84e38f"
server_key_version: "encryptionkey@0"
I20260807 15:14:24.986140 15416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:24.991355 15416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:24.994944 15416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:24.995177 15416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:25.003301 15416 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:36329
I20260807 15:14:25.003328 15508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:36329 every 8 connection(s)
I20260807 15:14:25.004384 15416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:25.007570 15509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:25.012109 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 15416
I20260807 15:14:25.012269 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:14:25.012605 16265 external_mini_cluster.cc:1428] Setting key 830eda13177af490ec2d1c5f7ffd0714
I20260807 15:14:25.014098 15509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 708d2dd2b93e420781b2bc2d1f9dc68e: Bootstrap starting.
I20260807 15:14:25.016671 15509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 708d2dd2b93e420781b2bc2d1f9dc68e: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:25.017599 15509 log.cc:824] T 00000000000000000000000000000000 P 708d2dd2b93e420781b2bc2d1f9dc68e: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:25.018277 15514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:25.014160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51081 (local address 127.15.226.126:36329)
0807 15:14:25.014628 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:25.014644 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:25.014707 (+ 63us) server_negotiation.cc:407] Connection header received
0807 15:14:25.015397 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:25.015417 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:25.015709 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:25.016075 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:25.016998 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:25.017003 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:25.017014 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:25.017025 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:25.017081 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:25.017100 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:25.017224 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:25.017327 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 15:14:25.017500 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":147,"threads_started":1}
I20260807 15:14:25.019727 15509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 708d2dd2b93e420781b2bc2d1f9dc68e: No bootstrap required, opened a new log
I20260807 15:14:25.021330 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:14:25.024820 15509 raft_consensus.cc:348] T 00000000000000000000000000000000 P 708d2dd2b93e420781b2bc2d1f9dc68e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "708d2dd2b93e420781b2bc2d1f9dc68e" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36329 } }
I20260807 15:14:25.025163 15509 raft_consensus.cc:374] T 00000000000000000000000000000000 P 708d2dd2b93e420781b2bc2d1f9dc68e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:25.025242 15509 raft_consensus.cc:729] T 00000000000000000000000000000000 P 708d2dd2b93e420781b2bc2d1f9dc68e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 708d2dd2b93e420781b2bc2d1f9dc68e, State: Initialized, Role: FOLLOWER
I20260807 15:14:25.025805 15514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:25.023976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34049 (local address 127.15.226.126:36329)
0807 15:14:25.024131 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:25.024136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:25.024345 (+ 209us) server_negotiation.cc:407] Connection header received
0807 15:14:25.024475 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:25.024477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:25.024510 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:25.024586 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:25.025396 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:25.025399 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:25.025401 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:25.025413 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:25.025439 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:25.025450 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:25.025554 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:25.025638 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 15:14:25.025682 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 15:14:25.025787 15509 consensus_queue.cc:260] T 00000000000000000000000000000000 P 708d2dd2b93e420781b2bc2d1f9dc68e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "708d2dd2b93e420781b2bc2d1f9dc68e" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36329 } }
I20260807 15:14:25.026002 15509 raft_consensus.cc:388] T 00000000000000000000000000000000 P 708d2dd2b93e420781b2bc2d1f9dc68e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:25.026077 15509 raft_consensus.cc:482] T 00000000000000000000000000000000 P 708d2dd2b93e420781b2bc2d1f9dc68e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:25.026202 15509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 708d2dd2b93e420781b2bc2d1f9dc68e [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:25.027498 15509 raft_consensus.cc:504] T 00000000000000000000000000000000 P 708d2dd2b93e420781b2bc2d1f9dc68e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "708d2dd2b93e420781b2bc2d1f9dc68e" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36329 } }
I20260807 15:14:25.027815 15509 leader_election.cc:302] T 00000000000000000000000000000000 P 708d2dd2b93e420781b2bc2d1f9dc68e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 708d2dd2b93e420781b2bc2d1f9dc68e; no voters:
I20260807 15:14:25.028086 15509 leader_election.cc:288] T 00000000000000000000000000000000 P 708d2dd2b93e420781b2bc2d1f9dc68e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:25.028260 15523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 708d2dd2b93e420781b2bc2d1f9dc68e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:25.028497 15523 raft_consensus.cc:686] T 00000000000000000000000000000000 P 708d2dd2b93e420781b2bc2d1f9dc68e [term 1 LEADER]: Becoming Leader. State: Replica: 708d2dd2b93e420781b2bc2d1f9dc68e, State: Running, Role: LEADER
I20260807 15:14:25.028859 15523 consensus_queue.cc:237] T 00000000000000000000000000000000 P 708d2dd2b93e420781b2bc2d1f9dc68e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "708d2dd2b93e420781b2bc2d1f9dc68e" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36329 } }
I20260807 15:14:25.029498 15509 sys_catalog.cc:564] T 00000000000000000000000000000000 P 708d2dd2b93e420781b2bc2d1f9dc68e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:25.030782 15524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 708d2dd2b93e420781b2bc2d1f9dc68e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "708d2dd2b93e420781b2bc2d1f9dc68e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "708d2dd2b93e420781b2bc2d1f9dc68e" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36329 } } }
I20260807 15:14:25.030934 15524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 708d2dd2b93e420781b2bc2d1f9dc68e [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:25.031344 15526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 708d2dd2b93e420781b2bc2d1f9dc68e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 708d2dd2b93e420781b2bc2d1f9dc68e. Latest consensus state: current_term: 1 leader_uuid: "708d2dd2b93e420781b2bc2d1f9dc68e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "708d2dd2b93e420781b2bc2d1f9dc68e" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 36329 } } }
I20260807 15:14:25.031484 15526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 708d2dd2b93e420781b2bc2d1f9dc68e [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:25.031409 15534 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:25.034940 15534 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:25.041205 15534 catalog_manager.cc:1347] Generated new cluster ID: 8cc6c5a8587b469399437cb87f0a47f0
I20260807 15:14:25.041313 15534 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:25.059122 15534 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:25.059526 15534 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:25.065631 15534 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 708d2dd2b93e420781b2bc2d1f9dc68e: Generated new TSK 0
I20260807 15:14:25.066393 15534 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:14:25.069368 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 15416
2026-08-07T15:14:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1654 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:25 dist-test-slave-3kk6 krb5kdc[15548](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:25 dist-test-slave-3kk6 krb5kdc[15548](info): set up 2 sockets
Aug 07 15:14:25 dist-test-slave-3kk6 krb5kdc[15548](info): commencing operation
krb5kdc: starting...
W20260807 15:14:26.337769 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:14:26 dist-test-slave-3kk6 krb5kdc[15548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:14:26.478089 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:44455
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:44247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:44455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:26.596544 15566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:26.596903 15566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:26.596997 15566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:26.601264 15566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:26.601394 15566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:26.601433 15566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:26.601476 15566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:26.601511 15566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:26.607183 15566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:44247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:44455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:44455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:26.608354 15566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:26.609463 15566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:26.618923 15576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:26.618937 15572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:26.618927 15573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:26.619089 15566 server_base.cc:1034] running on GCE node
I20260807 15:14:26.619956 15566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:26.621187 15566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:26.622398 15566 hybrid_clock.cc:648] HybridClock initialized: now 1786115666622368 us; error 45 us; skew 500 ppm
Aug 07 15:14:26 dist-test-slave-3kk6 krb5kdc[15548](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115666, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:26.625599 15566 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:14:26.626837 15566 webserver.cc:460] Webserver started at http://127.15.226.126:40307/ using document root <none> and password file <none>
I20260807 15:14:26.627456 15566 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:26.627554 15566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:26.627804 15566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:26.630046 15566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "d3ed1d098701493699a9fcb2875b44e2"
format_stamp: "Formatted at 2026-08-07 15:14:26 on dist-test-slave-3kk6"
server_key: "7ad5b191bdfef4a552394b9bad4c6586"
server_key_iv: "1a4143c6b290c60bf996c589fdab7098"
server_key_version: "encryptionkey@0"
I20260807 15:14:26.630699 15566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "d3ed1d098701493699a9fcb2875b44e2"
format_stamp: "Formatted at 2026-08-07 15:14:26 on dist-test-slave-3kk6"
server_key: "7ad5b191bdfef4a552394b9bad4c6586"
server_key_iv: "1a4143c6b290c60bf996c589fdab7098"
server_key_version: "encryptionkey@0"
I20260807 15:14:26.634804 15566 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 15:14:26.637603 15587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:26.639107 15566 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:14:26.639266 15566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "d3ed1d098701493699a9fcb2875b44e2"
format_stamp: "Formatted at 2026-08-07 15:14:26 on dist-test-slave-3kk6"
server_key: "7ad5b191bdfef4a552394b9bad4c6586"
server_key_iv: "1a4143c6b290c60bf996c589fdab7098"
server_key_version: "encryptionkey@0"
I20260807 15:14:26.639395 15566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:26.668457 15566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:26.671948 15566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:26.672187 15566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:26.680150 15566 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:44455
I20260807 15:14:26.680168 15664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:44455 every 8 connection(s)
I20260807 15:14:26.681216 15566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:26.684084 15666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:26.686623 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 15566
I20260807 15:14:26.686849 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:14:26.687144 16265 external_mini_cluster.cc:1428] Setting key 50ff9bbb97d4de8f781361b187664fac
I20260807 15:14:26.690292 15666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3ed1d098701493699a9fcb2875b44e2: Bootstrap starting.
I20260807 15:14:26.692621 15666 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3ed1d098701493699a9fcb2875b44e2: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:26.693388 15666 log.cc:824] T 00000000000000000000000000000000 P d3ed1d098701493699a9fcb2875b44e2: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:26.695685 15666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3ed1d098701493699a9fcb2875b44e2: No bootstrap required, opened a new log
Aug 07 15:14:26 dist-test-slave-3kk6 krb5kdc[15548](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:14:26.699146 15666 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3ed1d098701493699a9fcb2875b44e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3ed1d098701493699a9fcb2875b44e2" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44455 } }
I20260807 15:14:26.699407 15666 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3ed1d098701493699a9fcb2875b44e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:26.699491 15666 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3ed1d098701493699a9fcb2875b44e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3ed1d098701493699a9fcb2875b44e2, State: Initialized, Role: FOLLOWER
I20260807 15:14:26.699960 15666 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3ed1d098701493699a9fcb2875b44e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3ed1d098701493699a9fcb2875b44e2" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44455 } }
I20260807 15:14:26.700142 15666 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3ed1d098701493699a9fcb2875b44e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:26.700235 15666 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3ed1d098701493699a9fcb2875b44e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:26.700351 15666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3ed1d098701493699a9fcb2875b44e2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:26.701325 15666 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3ed1d098701493699a9fcb2875b44e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3ed1d098701493699a9fcb2875b44e2" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44455 } }
I20260807 15:14:26.701694 15666 leader_election.cc:302] T 00000000000000000000000000000000 P d3ed1d098701493699a9fcb2875b44e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3ed1d098701493699a9fcb2875b44e2; no voters:
I20260807 15:14:26.702113 15666 leader_election.cc:288] T 00000000000000000000000000000000 P d3ed1d098701493699a9fcb2875b44e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:26.702210 15671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3ed1d098701493699a9fcb2875b44e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:26.702462 15671 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3ed1d098701493699a9fcb2875b44e2 [term 1 LEADER]: Becoming Leader. State: Replica: d3ed1d098701493699a9fcb2875b44e2, State: Running, Role: LEADER
I20260807 15:14:26.702881 15671 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3ed1d098701493699a9fcb2875b44e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3ed1d098701493699a9fcb2875b44e2" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44455 } }
I20260807 15:14:26.703078 15669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:26.688703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60445 (local address 127.15.226.126:44455)
0807 15:14:26.689158 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:26.689173 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:26.689211 (+ 38us) server_negotiation.cc:407] Connection header received
0807 15:14:26.689853 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:26.689870 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:26.690213 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:26.690572 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:26.691474 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:26.692496 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:26.693453 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:26.693976 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:26.694151 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:26.697085 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:26.697090 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:26.697103 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:26.697130 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:26.699569 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:26.700090 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:26.700096 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:26.700102 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:26.700322 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:26.700623 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:26.700627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:26.700628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:26.701056 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:26.701286 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:26.701607 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 15:14:26.701956 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":158,"threads_started":1}
I20260807 15:14:26.703498 15666 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3ed1d098701493699a9fcb2875b44e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:26.704144 15671 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3ed1d098701493699a9fcb2875b44e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3ed1d098701493699a9fcb2875b44e2. Latest consensus state: current_term: 1 leader_uuid: "d3ed1d098701493699a9fcb2875b44e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3ed1d098701493699a9fcb2875b44e2" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44455 } } }
I20260807 15:14:26.704272 15671 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3ed1d098701493699a9fcb2875b44e2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:26.704708 15672 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3ed1d098701493699a9fcb2875b44e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3ed1d098701493699a9fcb2875b44e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3ed1d098701493699a9fcb2875b44e2" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44455 } } }
I20260807 15:14:26.704818 15672 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3ed1d098701493699a9fcb2875b44e2 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:26.706699 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260807 15:14:26.708808 15688 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d3ed1d098701493699a9fcb2875b44e2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:14:26.708950 15688 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:14:26.709095 15679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:26.709796 15679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:26.715898 15679 catalog_manager.cc:1347] Generated new cluster ID: 0547f40aecb34cba96fa13533cd872b6
I20260807 15:14:26.716001 15679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:26.722241 15669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:26.708874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43379 (local address 127.15.226.126:44455)
0807 15:14:26.709138 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:26.709144 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:26.709164 (+ 20us) server_negotiation.cc:407] Connection header received
0807 15:14:26.709335 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:26.709338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:26.709390 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:26.709473 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:26.710209 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:26.710804 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:26.711635 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:26.711973 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:26.712082 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:26.717433 (+ 5351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:26.717437 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:26.717441 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:26.717471 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:26.719256 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:26.719818 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:26.719821 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:26.719823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:26.719906 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:26.720207 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:26.720210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:26.720211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:26.720421 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:26.720536 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:26.721951 (+ 1415us) server_negotiation.cc:299] Negotiation successful
0807 15:14:26.722080 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183}
I20260807 15:14:26.743139 15679 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:26.744478 15679 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:26.758138 15679 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3ed1d098701493699a9fcb2875b44e2: Generated new TSK 0
I20260807 15:14:26.759040 15679 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:14:26.765221 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 15566
2026-08-07T15:14:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1695 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:26 dist-test-slave-3kk6 krb5kdc[15705](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:26 dist-test-slave-3kk6 krb5kdc[15705](info): set up 2 sockets
Aug 07 15:14:26 dist-test-slave-3kk6 krb5kdc[15705](info): commencing operation
krb5kdc: starting...
W20260807 15:14:28.006776 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:14:28 dist-test-slave-3kk6 krb5kdc[15705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:14:28.148343 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:40825
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:44955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:40825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:28.266142 15722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:28.266469 15722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:28.266548 15722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:28.270630 15722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:28.270754 15722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:28.270790 15722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:28.270828 15722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:28.270876 15722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:28.276283 15722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:44955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:40825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:40825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:28.277392 15722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:28.278443 15722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:28.287178 15733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:28.287196 15731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:28.287272 15722 server_base.cc:1034] running on GCE node
W20260807 15:14:28.287200 15729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:28.287992 15722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:28.289149 15722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:28.290364 15722 hybrid_clock.cc:648] HybridClock initialized: now 1786115668290343 us; error 37 us; skew 500 ppm
Aug 07 15:14:28 dist-test-slave-3kk6 krb5kdc[15705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115668, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:28.293452 15722 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:14:28.294492 15722 webserver.cc:460] Webserver started at http://127.15.226.126:42355/ using document root <none> and password file <none>
I20260807 15:14:28.295423 15722 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:28.295524 15722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:28.295890 15722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:28.298221 15722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "ea5d57ce71a24b56831fbab6d6bedba5"
format_stamp: "Formatted at 2026-08-07 15:14:28 on dist-test-slave-3kk6"
server_key: "e176d195dd7e6057e6c476a091be13fd"
server_key_iv: "2a9f2093a2cfa9bfb3b26614a136f0bd"
server_key_version: "encryptionkey@0"
I20260807 15:14:28.298924 15722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "ea5d57ce71a24b56831fbab6d6bedba5"
format_stamp: "Formatted at 2026-08-07 15:14:28 on dist-test-slave-3kk6"
server_key: "e176d195dd7e6057e6c476a091be13fd"
server_key_iv: "2a9f2093a2cfa9bfb3b26614a136f0bd"
server_key_version: "encryptionkey@0"
I20260807 15:14:28.303027 15722 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:14:28.305655 15742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:28.306988 15722 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 15:14:28.307144 15722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "ea5d57ce71a24b56831fbab6d6bedba5"
format_stamp: "Formatted at 2026-08-07 15:14:28 on dist-test-slave-3kk6"
server_key: "e176d195dd7e6057e6c476a091be13fd"
server_key_iv: "2a9f2093a2cfa9bfb3b26614a136f0bd"
server_key_version: "encryptionkey@0"
I20260807 15:14:28.307267 15722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:28.330096 15722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:28.334018 15722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:28.334271 15722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:28.341605 15722 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:40825
I20260807 15:14:28.341686 15816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:40825 every 8 connection(s)
I20260807 15:14:28.342744 15722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:28.345687 15821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:28.346769 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 15722
I20260807 15:14:28.346951 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:14:28.347280 16265 external_mini_cluster.cc:1428] Setting key cb5cfbbff7544a7dccee5c8abb9439d7
I20260807 15:14:28.352182 15821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea5d57ce71a24b56831fbab6d6bedba5: Bootstrap starting.
I20260807 15:14:28.354501 15821 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea5d57ce71a24b56831fbab6d6bedba5: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:28.355334 15821 log.cc:824] T 00000000000000000000000000000000 P ea5d57ce71a24b56831fbab6d6bedba5: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:28.356711 15825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:28.348792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41089 (local address 127.15.226.126:40825)
0807 15:14:28.349244 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:28.349260 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:28.349300 (+ 40us) server_negotiation.cc:407] Connection header received
0807 15:14:28.349960 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:28.349979 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:28.350328 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:28.350697 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:28.351140 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:28.352143 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:28.353145 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:28.353642 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:28.353822 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:28.354802 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:28.354808 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:28.354822 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:28.355221 (+ 399us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:28.355277 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:28.355299 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:28.355423 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:28.355597 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 15:14:28.355834 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":137,"threads_started":1}
I20260807 15:14:28.357707 15821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea5d57ce71a24b56831fbab6d6bedba5: No bootstrap required, opened a new log
I20260807 15:14:28.359760 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:14:28.361879 15821 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea5d57ce71a24b56831fbab6d6bedba5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea5d57ce71a24b56831fbab6d6bedba5" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 40825 } }
I20260807 15:14:28.362241 15821 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea5d57ce71a24b56831fbab6d6bedba5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:28.362332 15821 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea5d57ce71a24b56831fbab6d6bedba5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea5d57ce71a24b56831fbab6d6bedba5, State: Initialized, Role: FOLLOWER
I20260807 15:14:28.362908 15821 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea5d57ce71a24b56831fbab6d6bedba5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea5d57ce71a24b56831fbab6d6bedba5" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 40825 } }
I20260807 15:14:28.363075 15821 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea5d57ce71a24b56831fbab6d6bedba5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:28.363169 15821 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea5d57ce71a24b56831fbab6d6bedba5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:28.363304 15821 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea5d57ce71a24b56831fbab6d6bedba5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:28.364636 15821 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea5d57ce71a24b56831fbab6d6bedba5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea5d57ce71a24b56831fbab6d6bedba5" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 40825 } }
I20260807 15:14:28.365044 15821 leader_election.cc:302] T 00000000000000000000000000000000 P ea5d57ce71a24b56831fbab6d6bedba5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea5d57ce71a24b56831fbab6d6bedba5; no voters:
I20260807 15:14:28.365468 15821 leader_election.cc:288] T 00000000000000000000000000000000 P ea5d57ce71a24b56831fbab6d6bedba5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:28.365571 15834 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea5d57ce71a24b56831fbab6d6bedba5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:28.365761 15834 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea5d57ce71a24b56831fbab6d6bedba5 [term 1 LEADER]: Becoming Leader. State: Replica: ea5d57ce71a24b56831fbab6d6bedba5, State: Running, Role: LEADER
I20260807 15:14:28.366071 15834 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea5d57ce71a24b56831fbab6d6bedba5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea5d57ce71a24b56831fbab6d6bedba5" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 40825 } }
I20260807 15:14:28.366925 15821 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea5d57ce71a24b56831fbab6d6bedba5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:28.366966 15825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:28.361933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40251 (local address 127.15.226.126:40825)
0807 15:14:28.362084 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:28.362088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:28.362306 (+ 218us) server_negotiation.cc:407] Connection header received
0807 15:14:28.362452 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:28.362456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:28.362508 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:28.362597 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:28.362934 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:28.363669 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:28.364839 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:28.365114 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:28.365254 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:28.366256 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:28.366260 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:28.366262 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:28.366275 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:28.366303 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:28.366314 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:28.366375 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:28.366787 (+ 412us) server_negotiation.cc:299] Negotiation successful
0807 15:14:28.366849 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 15:14:28.368170 15835 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea5d57ce71a24b56831fbab6d6bedba5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea5d57ce71a24b56831fbab6d6bedba5. Latest consensus state: current_term: 1 leader_uuid: "ea5d57ce71a24b56831fbab6d6bedba5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea5d57ce71a24b56831fbab6d6bedba5" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 40825 } } }
I20260807 15:14:28.368336 15835 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea5d57ce71a24b56831fbab6d6bedba5 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:28.368762 15843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:28.369771 15836 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea5d57ce71a24b56831fbab6d6bedba5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea5d57ce71a24b56831fbab6d6bedba5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea5d57ce71a24b56831fbab6d6bedba5" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 40825 } } }
I20260807 15:14:28.369879 15836 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea5d57ce71a24b56831fbab6d6bedba5 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:28.372143 15843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:28.378065 15843 catalog_manager.cc:1347] Generated new cluster ID: 05172f85c6344b5fb253a65a4ce53469
I20260807 15:14:28.378159 15843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:28.384711 15843 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:28.385712 15843 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:28.391534 15843 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea5d57ce71a24b56831fbab6d6bedba5: Generated new TSK 0
I20260807 15:14:28.392376 15843 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:14:28.400439 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 15722
2026-08-07T15:14:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1633 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:28 dist-test-slave-3kk6 krb5kdc[15858](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:28 dist-test-slave-3kk6 krb5kdc[15858](info): set up 2 sockets
Aug 07 15:14:28 dist-test-slave-3kk6 krb5kdc[15858](info): commencing operation
krb5kdc: starting...
W20260807 15:14:29.661046 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:14:29 dist-test-slave-3kk6 krb5kdc[15858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:14:29.802970 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:44269
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:36801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:44269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:29.921123 15876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:29.921576 15876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:29.921674 15876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:29.925968 15876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:29.926092 15876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:29.926132 15876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:29.926170 15876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:29.926208 15876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:29.932092 15876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:36801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:44269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:44269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:29.933394 15876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:29.934497 15876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:29.943881 15883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:29.943886 15884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:29.943847 15886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:29.945052 15876 server_base.cc:1034] running on GCE node
I20260807 15:14:29.945842 15876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:29.947417 15876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:29.948688 15876 hybrid_clock.cc:648] HybridClock initialized: now 1786115669948666 us; error 43 us; skew 500 ppm
Aug 07 15:14:29 dist-test-slave-3kk6 krb5kdc[15858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115669, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:29.952631 15876 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:14:29.953895 15876 webserver.cc:460] Webserver started at http://127.15.226.126:45145/ using document root <none> and password file <none>
I20260807 15:14:29.954555 15876 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:29.954660 15876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:29.954970 15876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:29.957383 15876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "d315a14038914a8a9a5a0c88ff240d18"
format_stamp: "Formatted at 2026-08-07 15:14:29 on dist-test-slave-3kk6"
server_key: "cbcf155f630c33c786f5c509499fe8aa"
server_key_iv: "767ca1c602020bd16e65029b8e43dc41"
server_key_version: "encryptionkey@0"
I20260807 15:14:29.958086 15876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "d315a14038914a8a9a5a0c88ff240d18"
format_stamp: "Formatted at 2026-08-07 15:14:29 on dist-test-slave-3kk6"
server_key: "cbcf155f630c33c786f5c509499fe8aa"
server_key_iv: "767ca1c602020bd16e65029b8e43dc41"
server_key_version: "encryptionkey@0"
I20260807 15:14:29.962692 15876 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 15:14:29.965559 15898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:29.967052 15876 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:14:29.967247 15876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "d315a14038914a8a9a5a0c88ff240d18"
format_stamp: "Formatted at 2026-08-07 15:14:29 on dist-test-slave-3kk6"
server_key: "cbcf155f630c33c786f5c509499fe8aa"
server_key_iv: "767ca1c602020bd16e65029b8e43dc41"
server_key_version: "encryptionkey@0"
I20260807 15:14:29.967398 15876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:29.972749 15876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:29.976531 15876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:29.976789 15876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:29.985288 15876 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:44269
I20260807 15:14:29.985365 15963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:44269 every 8 connection(s)
I20260807 15:14:29.986454 15876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:29.989632 15964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:29.991285 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 15876
I20260807 15:14:29.991451 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:14:29.991803 16265 external_mini_cluster.cc:1428] Setting key e1e53f75492619edacdfef2363b5c280
I20260807 15:14:29.997347 15964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d315a14038914a8a9a5a0c88ff240d18: Bootstrap starting.
I20260807 15:14:29.997737 15967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:29.993455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58073 (local address 127.15.226.126:44269)
0807 15:14:29.993957 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:29.993976 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:29.994015 (+ 39us) server_negotiation.cc:407] Connection header received
0807 15:14:29.994732 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:29.994754 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:29.995065 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:29.995363 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:29.996322 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:29.996327 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:29.996339 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:29.996350 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:29.996411 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:29.996431 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:29.996548 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:29.996683 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 15:14:29.996872 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":152,"threads_started":1}
I20260807 15:14:29.999886 15964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d315a14038914a8a9a5a0c88ff240d18: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:30.000695 15964 log.cc:824] T 00000000000000000000000000000000 P d315a14038914a8a9a5a0c88ff240d18: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:30.000860 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:14:30.003309 15964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d315a14038914a8a9a5a0c88ff240d18: No bootstrap required, opened a new log
I20260807 15:14:30.005393 15967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:30.003282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46425 (local address 127.15.226.126:44269)
0807 15:14:30.003430 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:30.003434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:30.003725 (+ 291us) server_negotiation.cc:407] Connection header received
0807 15:14:30.003894 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:30.003897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:30.003932 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:30.004011 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:30.004804 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:30.004808 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:30.004810 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 15:14:30.004823 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:30.004851 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 15:14:30.004862 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:30.004936 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:30.005063 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 15:14:30.005098 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 15:14:30.007833 15964 raft_consensus.cc:348] T 00000000000000000000000000000000 P d315a14038914a8a9a5a0c88ff240d18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d315a14038914a8a9a5a0c88ff240d18" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44269 } }
I20260807 15:14:30.008118 15964 raft_consensus.cc:374] T 00000000000000000000000000000000 P d315a14038914a8a9a5a0c88ff240d18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 15:14:30.008188 15964 raft_consensus.cc:729] T 00000000000000000000000000000000 P d315a14038914a8a9a5a0c88ff240d18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d315a14038914a8a9a5a0c88ff240d18, State: Initialized, Role: FOLLOWER
I20260807 15:14:30.008634 15964 consensus_queue.cc:260] T 00000000000000000000000000000000 P d315a14038914a8a9a5a0c88ff240d18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d315a14038914a8a9a5a0c88ff240d18" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44269 } }
I20260807 15:14:30.008803 15964 raft_consensus.cc:388] T 00000000000000000000000000000000 P d315a14038914a8a9a5a0c88ff240d18 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 15:14:30.008906 15964 raft_consensus.cc:482] T 00000000000000000000000000000000 P d315a14038914a8a9a5a0c88ff240d18 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 15:14:30.009017 15964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d315a14038914a8a9a5a0c88ff240d18 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:30.010102 15964 raft_consensus.cc:504] T 00000000000000000000000000000000 P d315a14038914a8a9a5a0c88ff240d18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d315a14038914a8a9a5a0c88ff240d18" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44269 } }
I20260807 15:14:30.010485 15964 leader_election.cc:302] T 00000000000000000000000000000000 P d315a14038914a8a9a5a0c88ff240d18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d315a14038914a8a9a5a0c88ff240d18; no voters:
I20260807 15:14:30.010874 15964 leader_election.cc:288] T 00000000000000000000000000000000 P d315a14038914a8a9a5a0c88ff240d18 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 15:14:30.011080 15976 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d315a14038914a8a9a5a0c88ff240d18 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:30.011379 15976 raft_consensus.cc:686] T 00000000000000000000000000000000 P d315a14038914a8a9a5a0c88ff240d18 [term 1 LEADER]: Becoming Leader. State: Replica: d315a14038914a8a9a5a0c88ff240d18, State: Running, Role: LEADER
I20260807 15:14:30.011770 15976 consensus_queue.cc:237] T 00000000000000000000000000000000 P d315a14038914a8a9a5a0c88ff240d18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d315a14038914a8a9a5a0c88ff240d18" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44269 } }
I20260807 15:14:30.012387 15964 sys_catalog.cc:564] T 00000000000000000000000000000000 P d315a14038914a8a9a5a0c88ff240d18 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:30.013731 15980 sys_catalog.cc:455] T 00000000000000000000000000000000 P d315a14038914a8a9a5a0c88ff240d18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d315a14038914a8a9a5a0c88ff240d18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d315a14038914a8a9a5a0c88ff240d18" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44269 } } }
I20260807 15:14:30.013897 15980 sys_catalog.cc:458] T 00000000000000000000000000000000 P d315a14038914a8a9a5a0c88ff240d18 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:30.014087 15981 sys_catalog.cc:455] T 00000000000000000000000000000000 P d315a14038914a8a9a5a0c88ff240d18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d315a14038914a8a9a5a0c88ff240d18. Latest consensus state: current_term: 1 leader_uuid: "d315a14038914a8a9a5a0c88ff240d18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d315a14038914a8a9a5a0c88ff240d18" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 44269 } } }
I20260807 15:14:30.014273 15981 sys_catalog.cc:458] T 00000000000000000000000000000000 P d315a14038914a8a9a5a0c88ff240d18 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:30.014441 15992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:30.018038 15992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:30.025344 15992 catalog_manager.cc:1347] Generated new cluster ID: 28b43af5647b400194fe7af1c827d39c
I20260807 15:14:30.025444 15992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:30.038182 15992 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:30.038544 15992 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:30.047130 15992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d315a14038914a8a9a5a0c88ff240d18: Generated new TSK 0
I20260807 15:14:30.047966 15992 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:14:30.053400 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 15876
2026-08-07T15:14:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1656 ms)
[----------] 18 tests from AuthTokenIssuingTest (30028 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:30 dist-test-slave-3kk6 krb5kdc[16011](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:30 dist-test-slave-3kk6 krb5kdc[16011](info): set up 2 sockets
Aug 07 15:14:30 dist-test-slave-3kk6 krb5kdc[16011](info): commencing operation
krb5kdc: starting...
W20260807 15:14:31.300199 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:14:31 dist-test-slave-3kk6 krb5kdc[16011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:14:31.441573 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:46147
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:32815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:46147,127.15.226.125:43999,127.15.226.124:34659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:31.561196 16030 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:14:31.561537 16030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:31.561563 16030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:31.561655 16030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:31.566507 16030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:31.566601 16030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:31.566622 16030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:31.566640 16030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:31.566732 16030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:31.572435 16030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:32815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:46147,127.15.226.125:43999,127.15.226.124:34659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:46147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:31.573599 16030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:31.574728 16030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:31.583647 16041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:31.583721 16030 server_base.cc:1034] running on GCE node
W20260807 15:14:31.583648 16036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:31.583647 16037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:31.584685 16030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:31.585824 16030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:31.586992 16030 hybrid_clock.cc:648] HybridClock initialized: now 1786115671586972 us; error 36 us; skew 500 ppm
Aug 07 15:14:31 dist-test-slave-3kk6 krb5kdc[16011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115671, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:31.590092 16030 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:14:31.591163 16030 webserver.cc:460] Webserver started at http://127.15.226.126:46455/ using document root <none> and password file <none>
I20260807 15:14:31.591776 16030 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:31.591877 16030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:31.592130 16030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:31.594275 16030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "6532903ceed64d74b0b7352565973d75"
format_stamp: "Formatted at 2026-08-07 15:14:31 on dist-test-slave-3kk6"
server_key: "93e9b09bc79ef899d5fd6447e411f454"
server_key_iv: "d28cda07acd2e0bf900ae66f81740e65"
server_key_version: "encryptionkey@0"
I20260807 15:14:31.594954 16030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "6532903ceed64d74b0b7352565973d75"
format_stamp: "Formatted at 2026-08-07 15:14:31 on dist-test-slave-3kk6"
server_key: "93e9b09bc79ef899d5fd6447e411f454"
server_key_iv: "d28cda07acd2e0bf900ae66f81740e65"
server_key_version: "encryptionkey@0"
I20260807 15:14:31.598856 16030 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 15:14:31.601372 16048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:31.602778 16030 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 15:14:31.602931 16030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "6532903ceed64d74b0b7352565973d75"
format_stamp: "Formatted at 2026-08-07 15:14:31 on dist-test-slave-3kk6"
server_key: "93e9b09bc79ef899d5fd6447e411f454"
server_key_iv: "d28cda07acd2e0bf900ae66f81740e65"
server_key_version: "encryptionkey@0"
I20260807 15:14:31.603060 16030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:31.629357 16030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:31.632947 16030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:31.633193 16030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:31.641021 16030 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:46147
I20260807 15:14:31.641341 16116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:46147 every 8 connection(s)
I20260807 15:14:31.642119 16030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:31.645100 16118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:31.649104 16118 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46147 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:14:31.650574 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 16030
I20260807 15:14:31.650763 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:14:31.651072 16265 external_mini_cluster.cc:1428] Setting key b9c39ab1edb4d2b3ffd74e6dce3bde7e
Aug 07 15:14:31 dist-test-slave-3kk6 krb5kdc[16011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115671, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.226.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.125@KRBTEST.COM" created.
I20260807 15:14:31.666376 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:31.651662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46147 (local address 127.0.0.1:46442)
0807 15:14:31.652196 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:31.652252 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:31.653396 (+ 1144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:31.654162 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:31.654168 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:31.654752 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:31.655181 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:31.655204 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:31.656466 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:31.656473 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:31.657838 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:31.657853 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:31.658510 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:31.658516 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:31.658699 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:31.659349 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:31.659377 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:31.661217 (+ 1840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:31.663295 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:31.663306 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:31.663317 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:31.663557 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:31.663905 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:31.663910 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:31.663912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:31.664046 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:31.664826 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:31.664832 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:31.664975 (+ 143us) client_negotiation.cc:769] Sending connection context
0807 15:14:31.665172 (+ 197us) client_negotiation.cc:240] Negotiation successful
0807 15:14:31.665496 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":158,"threads_started":1}
I20260807 15:14:31.666385 16123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:31.652606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46442 (local address 127.15.226.126:46147)
0807 15:14:31.652958 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:31.652977 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:31.653034 (+ 57us) server_negotiation.cc:407] Connection header received
0807 15:14:31.653743 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:31.653748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:31.653938 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:31.654103 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:31.655366 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:31.656278 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:31.657970 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:31.658336 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:31.658466 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:31.661404 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:31.661410 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:31.661421 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:31.661454 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:31.663173 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:31.663676 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:31.663684 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:31.663692 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:31.663801 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:31.664155 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:31.664158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:31.664160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:31.664636 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:31.664813 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:31.665302 (+ 489us) server_negotiation.cc:299] Negotiation successful
0807 15:14:31.665648 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":104,"server-negotiator.queue_time_us":209,"thread_start_us":88,"threads_started":1}
I20260807 15:14:31.669265 16118 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:14:31.670195 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:31.669818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:43999 (local address 127.0.0.1:53746)
0807 15:14:31.669981 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:31.670093 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.125:43999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 15:14:31.670516 16051 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.226.125:43999: connect: Connection refused (error 111)
I20260807 15:14:31.671468 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:31.671253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:43999 (local address 127.0.0.1:53748)
0807 15:14:31.671362 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:31.671409 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.125:43999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 15:14:31.671741 16118 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.125:43999: Network error: Client connection negotiation failed: client connection to 127.15.226.125:43999: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.125.keytab.
Entry for principal kudu/127.15.226.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.125.keytab.
I20260807 15:14:31.683470 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.125:43999
--webserver_interface=127.15.226.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.125.keytab
--principal=kudu/127.15.226.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:32815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:46147,127.15.226.125:43999,127.15.226.124:34659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:14:31.721020 16118 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 } attempt: 1
I20260807 15:14:31.722043 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:31.721702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:43999 (local address 127.0.0.1:53758)
0807 15:14:31.721875 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:31.721962 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.125:43999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 15:14:31.722858 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:31.722594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:43999 (local address 127.0.0.1:53762)
0807 15:14:31.722747 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:31.722801 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.125:43999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 15:14:31.723008 16118 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.125:43999: Network error: Client connection negotiation failed: client connection to 127.15.226.125:43999: connect: Connection refused (error 111)
I20260807 15:14:31.791190 16118 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 } attempt: 2
I20260807 15:14:31.792148 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:31.791823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:43999 (local address 127.0.0.1:53774)
0807 15:14:31.791992 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:31.792070 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.125:43999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 15:14:31.792894 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:31.792663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:43999 (local address 127.0.0.1:53778)
0807 15:14:31.792788 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:31.792840 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.125:43999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 15:14:31.793035 16118 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.125:43999: Network error: Client connection negotiation failed: client connection to 127.15.226.125:43999: connect: Connection refused (error 111)
W20260807 15:14:31.801968 16128 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:14:31.802366 16128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:31.802443 16128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:31.802527 16128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:31.806740 16128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:31.806905 16128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:31.806947 16128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:31.807003 16128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:31.807044 16128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:31.812736 16128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:32815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:46147,127.15.226.125:43999,127.15.226.124:34659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.125.keytab
--principal=kudu/127.15.226.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.125:43999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.226.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:31.813979 16128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:31.815131 16128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:31.824302 16136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:31.824353 16133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:31.824353 16134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:31.824436 16128 server_base.cc:1034] running on GCE node
I20260807 15:14:31.825343 16128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:31.826584 16128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:31.827862 16128 hybrid_clock.cc:648] HybridClock initialized: now 1786115671827842 us; error 36 us; skew 500 ppm
Aug 07 15:14:31 dist-test-slave-3kk6 krb5kdc[16011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115671, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:31.840101 16128 init.cc:377] Logged in from keytab as kudu/127.15.226.125@KRBTEST.COM (short username kudu)
I20260807 15:14:31.841401 16128 webserver.cc:460] Webserver started at http://127.15.226.125:33977/ using document root <none> and password file <none>
I20260807 15:14:31.842042 16128 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:31.842142 16128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:31.842406 16128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:31.844907 16128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-1/data/instance:
uuid: "98ec48b7d46e498ba401addacc9082c9"
format_stamp: "Formatted at 2026-08-07 15:14:31 on dist-test-slave-3kk6"
server_key: "cb7a813ce2d776de0a09c09b4792b1f5"
server_key_iv: "24d27bfbe6061e37e07023cfc4306f51"
server_key_version: "encryptionkey@0"
I20260807 15:14:31.845541 16128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-1/wal/instance:
uuid: "98ec48b7d46e498ba401addacc9082c9"
format_stamp: "Formatted at 2026-08-07 15:14:31 on dist-test-slave-3kk6"
server_key: "cb7a813ce2d776de0a09c09b4792b1f5"
server_key_iv: "24d27bfbe6061e37e07023cfc4306f51"
server_key_version: "encryptionkey@0"
I20260807 15:14:31.849543 16128 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:14:31.852324 16147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:31.853631 16128 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 15:14:31.853825 16128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-1/wal
uuid: "98ec48b7d46e498ba401addacc9082c9"
format_stamp: "Formatted at 2026-08-07 15:14:31 on dist-test-slave-3kk6"
server_key: "cb7a813ce2d776de0a09c09b4792b1f5"
server_key_iv: "24d27bfbe6061e37e07023cfc4306f51"
server_key_version: "encryptionkey@0"
I20260807 15:14:31.853966 16128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:31.866868 16128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:31.869975 16128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:31.870293 16128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:31.878153 16128 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.125:43999
I20260807 15:14:31.878479 16228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.125:43999 every 8 connection(s)
I20260807 15:14:31.879312 16128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-1/data/info.pb
I20260807 15:14:31.881705 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 16128
I20260807 15:14:31.881811 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-1/wal/instance
I20260807 15:14:31.882130 16265 external_mini_cluster.cc:1428] Setting key e150ab16c8fd5cf42023eab16db89bdf
I20260807 15:14:31.882486 16229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:31.884217 16118 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 } attempt: 3
I20260807 15:14:31.887212 16229 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46147 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.226.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.124@KRBTEST.COM" created.
Aug 07 15:14:31 dist-test-slave-3kk6 krb5kdc[16011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115671, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for kudu/127.15.226.125@KRBTEST.COM
Aug 07 15:14:31 dist-test-slave-3kk6 krb5kdc[16011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115671, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.125@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:14:31.902423 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:31.884874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:43999 (local address 127.0.0.1:53782)
0807 15:14:31.885109 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:31.885131 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:31.885237 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:31.888317 (+ 3080us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:31.888320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:31.888551 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:31.888713 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:31.888722 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:31.890880 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:31.890883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:31.891592 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:31.891603 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:31.892473 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:31.892480 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:31.892615 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:31.893256 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:31.893295 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:31.897222 (+ 3927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:31.899917 (+ 2695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:31.899923 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:31.899928 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:31.900220 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:31.900688 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:31.900694 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:31.900699 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:31.900837 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:31.901903 (+ 1066us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:31.901906 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:31.902031 (+ 125us) client_negotiation.cc:769] Sending connection context
0807 15:14:31.902133 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 15:14:31.902284 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":25,"spinlock_wait_cycles":20864}
I20260807 15:14:31.903929 16235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:31.889434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46147 (local address 127.0.0.1:46448)
0807 15:14:31.889859 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:31.889911 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:31.890097 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:31.890437 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:31.890443 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:31.890981 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:31.891146 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:31.891154 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:31.892289 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:31.892292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:31.893196 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:31.893211 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:31.894041 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:31.894046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:31.894161 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:31.894739 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:31.894756 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:31.897904 (+ 3148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:31.900959 (+ 3055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:31.900970 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:31.900976 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:31.901260 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:31.901569 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:31.901572 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:31.901574 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:31.901665 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:31.902169 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:31.902178 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:31.902372 (+ 194us) client_negotiation.cc:769] Sending connection context
0807 15:14:31.902582 (+ 210us) client_negotiation.cc:240] Negotiation successful
0807 15:14:31.903046 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"spinlock_wait_cycles":6912,"thread_start_us":158,"threads_started":1}
I20260807 15:14:31.903934 16234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:31.885644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53782 (local address 127.15.226.125:43999)
0807 15:14:31.886176 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:31.886198 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:31.886248 (+ 50us) server_negotiation.cc:407] Connection header received
0807 15:14:31.887424 (+ 1176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:31.887452 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:31.887943 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:31.888354 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:31.888873 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:31.890712 (+ 1839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:31.891735 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:31.892334 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:31.892500 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:31.897396 (+ 4896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:31.897403 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:31.897416 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:31.897443 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:31.899782 (+ 2339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:31.900368 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:31.900378 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:31.900386 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:31.900556 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:31.900965 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:31.900971 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:31.900976 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:31.901764 (+ 788us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:31.901918 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:31.902885 (+ 967us) server_negotiation.cc:299] Negotiation successful
0807 15:14:31.903096 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":346,"spinlock_wait_cycles":2816,"thread_start_us":154,"threads_started":1}
I20260807 15:14:31.905090 16123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:31.889430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46448 (local address 127.15.226.126:46147)
0807 15:14:31.889564 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:31.889569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:31.890005 (+ 436us) server_negotiation.cc:407] Connection header received
0807 15:14:31.890228 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:31.890231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:31.890286 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:31.890380 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:31.891307 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:31.892144 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:31.893333 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:31.893716 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:31.893846 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:31.898139 (+ 4293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:31.898147 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:31.898151 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:31.898187 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:31.900825 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:31.901371 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:31.901375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:31.901377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:31.901468 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:31.901764 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:31.901768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:31.901770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:31.902043 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:31.902149 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:31.904749 (+ 2600us) server_negotiation.cc:299] Negotiation successful
0807 15:14:31.904898 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":56}
I20260807 15:14:31.906138 16229 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:14:31.906535 16118 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:14:31.907454 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:31.907092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34659 (local address 127.0.0.1:35652)
0807 15:14:31.907305 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:31.907360 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260807 15:14:31.908360 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:31.908104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34659 (local address 127.0.0.1:35668)
0807 15:14:31.908262 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:31.908308 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260807 15:14:31.908556 16118 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.124:34659: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34659: connect: Connection refused (error 111)
Aug 07 15:14:31 dist-test-slave-3kk6 krb5kdc[16011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115671, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.125@KRBTEST.COM for kudu/127.15.226.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.124.keytab.
Entry for principal kudu/127.15.226.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.124.keytab.
I20260807 15:14:31.915584 16235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:31.906787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:43999 (local address 127.0.0.1:53786)
0807 15:14:31.906947 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:31.906966 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:31.907049 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:31.907329 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:31.907335 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:31.907531 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:31.907647 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:31.907653 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:31.908658 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:31.908664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:31.909264 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:31.909272 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:31.909726 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:31.909730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:31.909822 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:31.910326 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:31.910337 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:31.911774 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:31.913591 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:31.913597 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:31.913600 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:31.913811 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:31.914147 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:31.914151 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:31.914152 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:31.914242 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:31.914781 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:31.914787 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:31.914994 (+ 207us) client_negotiation.cc:769] Sending connection context
0807 15:14:31.915165 (+ 171us) client_negotiation.cc:240] Negotiation successful
0807 15:14:31.915361 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":2}
I20260807 15:14:31.915584 16234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:31.906962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53786 (local address 127.15.226.125:43999)
0807 15:14:31.907093 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:31.907100 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:31.907113 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:14:31.907154 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:31.907160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:31.907206 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:31.907294 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:31.907848 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:31.908560 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:31.909390 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:31.909617 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:31.909728 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:31.911989 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:31.911992 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:31.911994 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:31.912015 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:31.913468 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:31.913915 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:31.913919 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:31.913921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:31.914043 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:31.914342 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:31.914347 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:31.914348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:31.914572 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:31.914769 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:31.915214 (+ 445us) server_negotiation.cc:299] Negotiation successful
0807 15:14:31.915443 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":124,"server-negotiator.queue_time_us":50}
I20260807 15:14:31.916363 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.124:34659
--webserver_interface=127.15.226.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.124.keytab
--principal=kudu/127.15.226.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:32815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:46147,127.15.226.125:43999,127.15.226.124:34659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:14:31.916534 16229 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:14:31.917408 16235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:31.917025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34659 (local address 127.0.0.1:35682)
0807 15:14:31.917191 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:31.917312 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 15:14:31.917732 16155 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34659: connect: Connection refused (error 111)
I20260807 15:14:31.918849 16235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:31.918563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34659 (local address 127.0.0.1:35684)
0807 15:14:31.918745 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:31.918792 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 15:14:31.919097 16229 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.124:34659: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34659: connect: Connection refused (error 111)
I20260807 15:14:31.939805 16118 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } attempt: 1
I20260807 15:14:31.940960 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:31.940554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34659 (local address 127.0.0.1:35698)
0807 15:14:31.940768 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:31.940866 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260807 15:14:31.942008 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:31.941630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34659 (local address 127.0.0.1:35700)
0807 15:14:31.941862 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:31.941938 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260807 15:14:31.942248 16118 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.124:34659: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34659: connect: Connection refused (error 111)
I20260807 15:14:31.968410 16229 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } attempt: 1
I20260807 15:14:31.969502 16235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:31.969154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34659 (local address 127.0.0.1:35716)
0807 15:14:31.969333 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:31.969407 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 15:14:31.970368 16235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:31.970123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34659 (local address 127.0.0.1:35720)
0807 15:14:31.970245 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:31.970300 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 15:14:31.970570 16229 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.124:34659: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34659: connect: Connection refused (error 111)
I20260807 15:14:32.017479 16118 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } attempt: 2
I20260807 15:14:32.018450 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:32.018110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34659 (local address 127.0.0.1:35732)
0807 15:14:32.018282 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:32.018359 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 15:14:32.019374 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:32.019160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34659 (local address 127.0.0.1:35742)
0807 15:14:32.019268 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:32.019318 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 15:14:32.019565 16118 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.124:34659: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34659: connect: Connection refused (error 111)
I20260807 15:14:32.038748 16229 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } attempt: 2
I20260807 15:14:32.039712 16235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:32.039373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34659 (local address 127.0.0.1:35754)
0807 15:14:32.039555 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:32.039629 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 15:14:32.040652 16235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:32.040420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34659 (local address 127.0.0.1:35756)
0807 15:14:32.040538 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:32.040596 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 15:14:32.040827 16229 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.124:34659: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34659: connect: Connection refused (error 111)
W20260807 15:14:32.041366 16239 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:14:32.041688 16239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:32.041746 16239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:32.041837 16239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:32.045980 16239 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:32.046034 16239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:32.046056 16239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:32.046103 16239 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:32.046144 16239 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:32.051558 16239 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:32815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:46147,127.15.226.125:43999,127.15.226.124:34659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.124.keytab
--principal=kudu/127.15.226.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.124:34659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.226.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:32.052749 16239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:32.053834 16239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:32.062997 16245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:32.063028 16249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:32.062997 16247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:32.063264 16239 server_base.cc:1034] running on GCE node
I20260807 15:14:32.064013 16239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:32.065296 16239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:32.066490 16239 hybrid_clock.cc:648] HybridClock initialized: now 1786115672066465 us; error 51 us; skew 500 ppm
Aug 07 15:14:32 dist-test-slave-3kk6 krb5kdc[16011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115672, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:32.069558 16239 init.cc:377] Logged in from keytab as kudu/127.15.226.124@KRBTEST.COM (short username kudu)
I20260807 15:14:32.070492 16239 webserver.cc:460] Webserver started at http://127.15.226.124:34929/ using document root <none> and password file <none>
I20260807 15:14:32.071165 16239 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:32.071266 16239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:32.071518 16239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:32.073647 16239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-2/data/instance:
uuid: "b97ba2c0f46d42c29a1743c8108bd016"
format_stamp: "Formatted at 2026-08-07 15:14:32 on dist-test-slave-3kk6"
server_key: "4bc65d80ea8725ae79541634a08f7e34"
server_key_iv: "31998c77b3033189b4e31d0c2a899bd1"
server_key_version: "encryptionkey@0"
I20260807 15:14:32.074263 16239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-2/wal/instance:
uuid: "b97ba2c0f46d42c29a1743c8108bd016"
format_stamp: "Formatted at 2026-08-07 15:14:32 on dist-test-slave-3kk6"
server_key: "4bc65d80ea8725ae79541634a08f7e34"
server_key_iv: "31998c77b3033189b4e31d0c2a899bd1"
server_key_version: "encryptionkey@0"
I20260807 15:14:32.078181 16239 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 15:14:32.080751 16258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:32.082082 16239 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 15:14:32.082230 16239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-2/wal
uuid: "b97ba2c0f46d42c29a1743c8108bd016"
format_stamp: "Formatted at 2026-08-07 15:14:32 on dist-test-slave-3kk6"
server_key: "4bc65d80ea8725ae79541634a08f7e34"
server_key_iv: "31998c77b3033189b4e31d0c2a899bd1"
server_key_version: "encryptionkey@0"
I20260807 15:14:32.082352 16239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:32.094002 16239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:32.098416 16239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:32.098661 16239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:32.108100 16239 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.124:34659
I20260807 15:14:32.108412 16337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.124:34659 every 8 connection(s)
I20260807 15:14:32.109207 16239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-2/data/info.pb
I20260807 15:14:32.112326 16338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:32.116273 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 16239
I20260807 15:14:32.116415 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786115365131938-16265-0/minicluster-data/master-2/wal/instance
I20260807 15:14:32.116726 16265 external_mini_cluster.cc:1428] Setting key 61ec77aac0ad0f84537e3c1e8aa5541e
I20260807 15:14:32.116608 16338 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46147 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:14:32.118731 16118 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } attempt: 3
Aug 07 15:14:32 dist-test-slave-3kk6 krb5kdc[16011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115671, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for kudu/127.15.226.124@KRBTEST.COM
Aug 07 15:14:32 dist-test-slave-3kk6 krb5kdc[16011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115672, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.124@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
Aug 07 15:14:32 dist-test-slave-3kk6 krb5kdc[16011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.124@KRBTEST.COM
I20260807 15:14:32.132000 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:32.119169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34659 (local address 127.0.0.1:35776)
0807 15:14:32.119413 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:32.119440 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:32.119518 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:32.120515 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:32.120518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:32.120752 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:32.120871 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:32.120879 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.122553 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.122559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:32.123533 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:32.123546 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.124420 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.124426 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:32.124527 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:32.125051 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:32.125069 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:32.126905 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:32.129791 (+ 2886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:32.129794 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:32.129797 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:32.129992 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:32.130372 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:32.130375 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:32.130377 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:32.130466 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:32.131373 (+ 907us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:32.131376 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:32.131471 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 15:14:32.131595 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 15:14:32.131787 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":1}
I20260807 15:14:32.132014 16229 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } attempt: 3
I20260807 15:14:32.133333 16345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:32.119607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35776 (local address 127.15.226.124:34659)
0807 15:14:32.119870 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:32.119876 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:32.119888 (+ 12us) server_negotiation.cc:407] Connection header received
0807 15:14:32.120028 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:32.120032 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:32.120304 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:32.120994 (+ 690us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:32.121115 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.122417 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.123666 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.124268 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.124733 (+ 465us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:32.127159 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:32.127165 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:32.127177 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:32.127215 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:32.129637 (+ 2422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:32.130114 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:32.130120 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:32.130122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:32.130262 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:32.130569 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:32.130572 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:32.130574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:32.131238 (+ 664us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:32.131387 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:32.131807 (+ 420us) server_negotiation.cc:299] Negotiation successful
0807 15:14:32.132173 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":205,"spinlock_wait_cycles":19328,"thread_start_us":91,"threads_started":1}
I20260807 15:14:32.134202 16123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:32.118894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46456 (local address 127.15.226.126:46147)
0807 15:14:32.119035 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:32.119040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:32.119418 (+ 378us) server_negotiation.cc:407] Connection header received
0807 15:14:32.120267 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:32.120270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:32.120324 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:32.120421 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:32.121562 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.122323 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.123469 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.123770 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.123875 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:32.127739 (+ 3864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:32.127745 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:32.127750 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:32.127784 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:32.129637 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:32.130735 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:32.130738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:32.130740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:32.130821 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:32.131148 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:32.131152 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:32.131154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:32.131501 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:32.131618 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:32.133977 (+ 2359us) server_negotiation.cc:299] Negotiation successful
0807 15:14:32.134083 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 15:14:32.133613 16343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:32.118254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35766 (local address 127.15.226.124:34659)
0807 15:14:32.118748 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:32.118768 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:32.119082 (+ 314us) server_negotiation.cc:407] Connection header received
0807 15:14:32.120011 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:32.120030 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:32.120304 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:32.120474 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:32.121455 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.122478 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.123848 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.124265 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.124385 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:32.128218 (+ 3833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:32.128223 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:32.128228 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:32.128262 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:32.130951 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:32.131489 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:32.131491 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:32.131492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:32.131586 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:32.131937 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:32.131940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:32.131942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:32.132419 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:32.132606 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:32.133247 (+ 641us) server_negotiation.cc:299] Negotiation successful
0807 15:14:32.133385 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":159,"server-negotiator.queue_time_us":308,"spinlock_wait_cycles":9344,"thread_start_us":131,"threads_started":1}
I20260807 15:14:32.134266 16344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:32.118816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:46147 (local address 127.0.0.1:46456)
0807 15:14:32.119269 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:32.119331 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:32.120026 (+ 695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:32.120497 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:32.120503 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:32.120979 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:32.121405 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:32.121421 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.122461 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.122467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:32.123332 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:32.123342 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.124052 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.124056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:32.124252 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:32.124989 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:32.125014 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:32.127524 (+ 2510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:32.129774 (+ 2250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:32.130106 (+ 332us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:32.130120 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:32.130563 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:32.130930 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:32.130934 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:32.130937 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:32.131049 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:32.133650 (+ 2601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:32.133655 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:32.133755 (+ 100us) client_negotiation.cc:769] Sending connection context
0807 15:14:32.133907 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 15:14:32.134086 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":48,"spinlock_wait_cycles":4352,"thread_start_us":160,"threads_started":1}
I20260807 15:14:32.136317 16338 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:14:32.139439 16118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75: Bootstrap starting.
Aug 07 15:14:32 dist-test-slave-3kk6 krb5kdc[16011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115671, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.125@KRBTEST.COM for kudu/127.15.226.124@KRBTEST.COM
Aug 07 15:14:32 dist-test-slave-3kk6 krb5kdc[16011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115672, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.124@KRBTEST.COM for kudu/127.15.226.125@KRBTEST.COM
I20260807 15:14:32.142968 16118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:32.144172 16118 log.cc:824] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:32.144608 16235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:32.132493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34659 (local address 127.0.0.1:35788)
0807 15:14:32.132774 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:32.132790 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:32.132875 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:32.133404 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:32.133408 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:32.133609 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:32.133740 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:32.133748 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.135896 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.135900 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:32.136702 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:32.136721 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.137909 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.137913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:32.138033 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:32.138761 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:32.138788 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:32.140485 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:32.142728 (+ 2243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:32.142732 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:32.142734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:32.142983 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:32.143308 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:32.143311 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:32.143313 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:32.143407 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:32.143898 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:32.143904 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:32.144018 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 15:14:32.144110 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 15:14:32.144419 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":3}
I20260807 15:14:32.144767 16347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:32.132778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35788 (local address 127.15.226.124:34659)
0807 15:14:32.133122 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:32.133131 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:32.133144 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:14:32.133175 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:32.133178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:32.133217 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:32.133350 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:32.133866 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.135757 (+ 1891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.137257 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.137767 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.137891 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:32.140639 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:32.140643 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:32.140646 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:32.140670 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:32.142554 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:32.143111 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:32.143114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:32.143116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:32.143203 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:32.143530 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:32.143533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:32.143534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:32.143775 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:32.144287 (+ 512us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:32.144370 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 15:14:32.144513 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":145,"threads_started":1}
Aug 07 15:14:32 dist-test-slave-3kk6 krb5kdc[16011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:14:32.147835 16118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75: No bootstrap required, opened a new log
I20260807 15:14:32.148869 16234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:32.137125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53790 (local address 127.15.226.125:43999)
0807 15:14:32.137233 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:32.137239 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:32.137252 (+ 13us) server_negotiation.cc:407] Connection header received
0807 15:14:32.137283 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:32.137286 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:32.137326 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:32.137392 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:32.137913 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.139055 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.140310 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.140641 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.140730 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:32.143418 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:32.143422 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:32.143425 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:32.143454 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:32.145564 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:32.146122 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:32.146126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:32.146128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:32.146239 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:32.147093 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:32.147097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:32.147099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:32.147736 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:32.147837 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:32.148550 (+ 713us) server_negotiation.cc:299] Negotiation successful
0807 15:14:32.148675 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":336,"server-negotiator.queue_time_us":31}
I20260807 15:14:32.149299 16229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98ec48b7d46e498ba401addacc9082c9: Bootstrap starting.
I20260807 15:14:32.149405 16344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:32.136841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:43999 (local address 127.0.0.1:53790)
0807 15:14:32.136953 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:32.136967 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:32.137041 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:32.137491 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:32.137494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:32.137665 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:32.137779 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:32.137786 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.139222 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.139226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:32.140056 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:32.140069 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.140903 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.140907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:32.141037 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:32.141503 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:32.141519 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:32.143222 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:32.145710 (+ 2488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:32.145714 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:32.145718 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:32.145998 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:32.146359 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:32.146363 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:32.146365 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:32.146462 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:32.147903 (+ 1441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:32.147908 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:32.148325 (+ 417us) client_negotiation.cc:769] Sending connection context
0807 15:14:32.149065 (+ 740us) client_negotiation.cc:240] Negotiation successful
0807 15:14:32.149228 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 15:14:32.150714 16123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:32.137545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46458 (local address 127.15.226.126:46147)
0807 15:14:32.137689 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:32.137695 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:32.138891 (+ 1196us) server_negotiation.cc:407] Connection header received
0807 15:14:32.139090 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:32.139093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:32.139137 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:32.139188 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:32.140346 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.141871 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.143100 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.143442 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.143520 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:32.147060 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:32.147067 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:32.147073 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:32.147101 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:32.148925 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:32.149434 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:32.149439 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:32.149441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:32.149531 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:32.149848 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:32.149851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:32.149853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:32.150034 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:32.150245 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:32.150419 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 15:14:32.150544 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 15:14:32.151414 16338 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:14:32.153168 16118 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532903ceed64d74b0b7352565973d75" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46147 } } peers { permanent_uuid: "98ec48b7d46e498ba401addacc9082c9" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 } } peers { permanent_uuid: "b97ba2c0f46d42c29a1743c8108bd016" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } }
I20260807 15:14:32.153455 16118 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6532903ceed64d74b0b7352565973d75, State: Initialized, Role: FOLLOWER
I20260807 15:14:32.153548 16229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98ec48b7d46e498ba401addacc9082c9: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:32.153925 16118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532903ceed64d74b0b7352565973d75" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46147 } } peers { permanent_uuid: "98ec48b7d46e498ba401addacc9082c9" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 } } peers { permanent_uuid: "b97ba2c0f46d42c29a1743c8108bd016" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } }
I20260807 15:14:32.154565 16229 log.cc:824] T 00000000000000000000000000000000 P 98ec48b7d46e498ba401addacc9082c9: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:32.154547 16351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532903ceed64d74b0b7352565973d75" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46147 } } peers { permanent_uuid: "98ec48b7d46e498ba401addacc9082c9" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 } } peers { permanent_uuid: "b97ba2c0f46d42c29a1743c8108bd016" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } } }
I20260807 15:14:32.154706 16351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:14:32.155644 16118 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:32.157290 16229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98ec48b7d46e498ba401addacc9082c9: No bootstrap required, opened a new log
Aug 07 15:14:32 dist-test-slave-3kk6 krb5kdc[16011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.125@KRBTEST.COM
Aug 07 15:14:32 dist-test-slave-3kk6 krb5kdc[16011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115672, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.124@KRBTEST.COM for kudu/127.15.226.124@KRBTEST.COM
I20260807 15:14:32.162580 16229 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98ec48b7d46e498ba401addacc9082c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532903ceed64d74b0b7352565973d75" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46147 } } peers { permanent_uuid: "98ec48b7d46e498ba401addacc9082c9" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 } } peers { permanent_uuid: "b97ba2c0f46d42c29a1743c8108bd016" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } }
I20260807 15:14:32.162930 16234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:32.151933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53798 (local address 127.15.226.125:43999)
0807 15:14:32.152138 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:32.152144 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:32.152311 (+ 167us) server_negotiation.cc:407] Connection header received
0807 15:14:32.152432 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:32.152435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:32.152474 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:32.152573 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:32.153167 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.153821 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.154923 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.155184 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.155291 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:32.158563 (+ 3272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:32.158567 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:32.158570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:32.158596 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:32.160669 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:32.161354 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:32.161357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:32.161359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:32.161454 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:32.161794 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:32.161797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:32.161799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:32.162015 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:32.162166 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:32.162553 (+ 387us) server_negotiation.cc:299] Negotiation successful
0807 15:14:32.162763 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260807 15:14:32.162916 16229 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98ec48b7d46e498ba401addacc9082c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98ec48b7d46e498ba401addacc9082c9, State: Initialized, Role: FOLLOWER
I20260807 15:14:32.163760 16229 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98ec48b7d46e498ba401addacc9082c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532903ceed64d74b0b7352565973d75" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46147 } } peers { permanent_uuid: "98ec48b7d46e498ba401addacc9082c9" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 } } peers { permanent_uuid: "b97ba2c0f46d42c29a1743c8108bd016" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } }
I20260807 15:14:32.165377 16229 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98ec48b7d46e498ba401addacc9082c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:32.165617 16344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:32.151907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34659 (local address 127.0.0.1:35794)
0807 15:14:32.155284 (+ 3377us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:32.155302 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:32.155427 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:32.155773 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:32.155778 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:32.155970 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:32.156063 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:32.156073 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.157318 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.157321 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:32.158224 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:32.158237 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.159103 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.159107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:32.159209 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:32.159657 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:32.159672 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:32.161740 (+ 2068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:32.164061 (+ 2321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:32.164067 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:32.164072 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:32.164324 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:32.164598 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:32.164601 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:32.164602 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:32.164675 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:32.165129 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:32.165132 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:32.165216 (+ 84us) client_negotiation.cc:769] Sending connection context
0807 15:14:32.165317 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 15:14:32.165463 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3293}
I20260807 15:14:32.166244 16347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:32.152122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35794 (local address 127.15.226.124:34659)
0807 15:14:32.152223 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:32.152227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:32.155364 (+ 3137us) server_negotiation.cc:407] Connection header received
0807 15:14:32.155525 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:32.155528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:32.155565 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:32.155614 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:32.156233 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.157111 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.158427 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.158855 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.158946 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:32.161891 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:32.161895 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:32.161898 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:32.161922 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:32.163927 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:32.164434 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:32.164437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:32.164438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:32.164511 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:32.164780 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:32.164782 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:32.164784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:32.164990 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:32.165095 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:32.165947 (+ 852us) server_negotiation.cc:299] Negotiation successful
0807 15:14:32.166096 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260807 15:14:32.168275 16370 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:14:32.168376 16370 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:14:32.161702 16030 master_runner.cc:186] Error getting master registration for 127.15.226.125:43999: OK, instance_id { permanent_uuid: "98ec48b7d46e498ba401addacc9082c9" instance_seqno: 1786115671875988 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:14:32.168982 16030 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:14:32.169771 16338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b97ba2c0f46d42c29a1743c8108bd016: Bootstrap starting.
I20260807 15:14:32.170446 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:14:32.171965 16083 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "6532903ceed64d74b0b7352565973d75"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46458
I20260807 15:14:32.172290 16083 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
W20260807 15:14:32.172349 16391 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 98ec48b7d46e498ba401addacc9082c9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260807 15:14:32.172410 16083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75 [term 0 FOLLOWER]: Advancing to term 1
W20260807 15:14:32.172436 16391 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:14:32.172796 16338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b97ba2c0f46d42c29a1743c8108bd016: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:32.173741 16338 log.cc:824] T 00000000000000000000000000000000 P b97ba2c0f46d42c29a1743c8108bd016: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:32.173921 16083 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532903ceed64d74b0b7352565973d75" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46147 } } peers { permanent_uuid: "98ec48b7d46e498ba401addacc9082c9" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 } } peers { permanent_uuid: "b97ba2c0f46d42c29a1743c8108bd016" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } }
I20260807 15:14:32.175083 16372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98ec48b7d46e498ba401addacc9082c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532903ceed64d74b0b7352565973d75" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46147 } } peers { permanent_uuid: "98ec48b7d46e498ba401addacc9082c9" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 } } peers { permanent_uuid: "b97ba2c0f46d42c29a1743c8108bd016" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } } }
I20260807 15:14:32.175247 16372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98ec48b7d46e498ba401addacc9082c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:14:32.175884 16195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6532903ceed64d74b0b7352565973d75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "98ec48b7d46e498ba401addacc9082c9"
I20260807 15:14:32.176095 16083 leader_election.cc:288] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75 [CANDIDATE]: Term 1 election: Requested vote from peers 98ec48b7d46e498ba401addacc9082c9 (127.15.226.125:43999), b97ba2c0f46d42c29a1743c8108bd016 (127.15.226.124:34659)
I20260807 15:14:32.176152 16195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98ec48b7d46e498ba401addacc9082c9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:32.177785 16195 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 98ec48b7d46e498ba401addacc9082c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6532903ceed64d74b0b7352565973d75 in term 1.
I20260807 15:14:32.178258 16051 leader_election.cc:302] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6532903ceed64d74b0b7352565973d75, 98ec48b7d46e498ba401addacc9082c9; no voters:
I20260807 15:14:32.178586 16351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:32.178884 16351 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75 [term 1 LEADER]: Becoming Leader. State: Replica: 6532903ceed64d74b0b7352565973d75, State: Running, Role: LEADER
I20260807 15:14:32.179263 16351 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532903ceed64d74b0b7352565973d75" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46147 } } peers { permanent_uuid: "98ec48b7d46e498ba401addacc9082c9" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 } } peers { permanent_uuid: "b97ba2c0f46d42c29a1743c8108bd016" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } }
I20260807 15:14:32.181319 16351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6532903ceed64d74b0b7352565973d75. Latest consensus state: current_term: 1 leader_uuid: "6532903ceed64d74b0b7352565973d75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532903ceed64d74b0b7352565973d75" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46147 } } peers { permanent_uuid: "98ec48b7d46e498ba401addacc9082c9" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 } } peers { permanent_uuid: "b97ba2c0f46d42c29a1743c8108bd016" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } } }
I20260807 15:14:32.181424 16351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:32.180738 16128 master_runner.cc:186] Error getting master registration for 127.15.226.124:34659: OK, instance_id { permanent_uuid: "b97ba2c0f46d42c29a1743c8108bd016" instance_seqno: 1786115672105781 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:14:32.184376 16128 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:14:32.187041 16402 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:32.187861 16338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b97ba2c0f46d42c29a1743c8108bd016: No bootstrap required, opened a new log
I20260807 15:14:32.188135 16402 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:32.193504 16338 raft_consensus.cc:348] T 00000000000000000000000000000000 P b97ba2c0f46d42c29a1743c8108bd016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532903ceed64d74b0b7352565973d75" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46147 } } peers { permanent_uuid: "98ec48b7d46e498ba401addacc9082c9" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 } } peers { permanent_uuid: "b97ba2c0f46d42c29a1743c8108bd016" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } }
I20260807 15:14:32.193840 16338 raft_consensus.cc:729] T 00000000000000000000000000000000 P b97ba2c0f46d42c29a1743c8108bd016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b97ba2c0f46d42c29a1743c8108bd016, State: Initialized, Role: FOLLOWER
I20260807 15:14:32.194236 16338 consensus_queue.cc:260] T 00000000000000000000000000000000 P b97ba2c0f46d42c29a1743c8108bd016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532903ceed64d74b0b7352565973d75" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46147 } } peers { permanent_uuid: "98ec48b7d46e498ba401addacc9082c9" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 } } peers { permanent_uuid: "b97ba2c0f46d42c29a1743c8108bd016" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } }
I20260807 15:14:32.195806 16338 sys_catalog.cc:564] T 00000000000000000000000000000000 P b97ba2c0f46d42c29a1743c8108bd016 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:32.196650 16298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6532903ceed64d74b0b7352565973d75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b97ba2c0f46d42c29a1743c8108bd016"
I20260807 15:14:32.196955 16298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b97ba2c0f46d42c29a1743c8108bd016 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:32.198875 16298 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b97ba2c0f46d42c29a1743c8108bd016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6532903ceed64d74b0b7352565973d75 in term 1.
I20260807 15:14:32.201251 16234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:32.185173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53806 (local address 127.15.226.125:43999)
0807 15:14:32.185340 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:32.185345 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:32.185361 (+ 16us) server_negotiation.cc:407] Connection header received
0807 15:14:32.185660 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:32.185663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:32.185728 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:32.185786 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:32.187681 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.188433 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.190305 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.190626 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.190764 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:32.194161 (+ 3397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:32.194164 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:32.194167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:32.194198 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:32.196453 (+ 2255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:32.199749 (+ 3296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:32.199754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:32.199756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:32.199867 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:32.200211 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:32.200215 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:32.200216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:32.200453 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:32.200562 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:32.200958 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 15:14:32.201094 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 15:14:32.203310 16406 sys_catalog.cc:455] T 00000000000000000000000000000000 P b97ba2c0f46d42c29a1743c8108bd016 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532903ceed64d74b0b7352565973d75" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46147 } } peers { permanent_uuid: "98ec48b7d46e498ba401addacc9082c9" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 } } peers { permanent_uuid: "b97ba2c0f46d42c29a1743c8108bd016" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } } }
I20260807 15:14:32.203483 16406 sys_catalog.cc:458] T 00000000000000000000000000000000 P b97ba2c0f46d42c29a1743c8108bd016 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:14:32.204183 16347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:32.185664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35798 (local address 127.15.226.124:34659)
0807 15:14:32.185795 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:32.185799 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:32.191780 (+ 5981us) server_negotiation.cc:407] Connection header received
0807 15:14:32.192003 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:32.192009 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:32.192069 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:32.192129 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:32.193212 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.193967 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.195214 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.195606 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.195699 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:32.197506 (+ 1807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:32.197510 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:32.197513 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:32.197538 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:32.199355 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:32.201509 (+ 2154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:32.201513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:32.201514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:32.201607 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:32.203185 (+ 1578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:32.203191 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:32.203205 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:32.203428 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:32.203537 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:32.203889 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 15:14:32.204013 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 15:14:32.208959 16298 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b97ba2c0f46d42c29a1743c8108bd016 [term 1 FOLLOWER]: Refusing update from remote peer 6532903ceed64d74b0b7352565973d75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:32.213361 16123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:32.200019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46464 (local address 127.15.226.126:46147)
0807 15:14:32.200150 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:32.200155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:32.202058 (+ 1903us) server_negotiation.cc:407] Connection header received
0807 15:14:32.202256 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:32.202260 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:32.202314 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:32.202369 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:32.203221 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.203984 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.205170 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:32.205464 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:32.205545 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:32.209215 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:32.209220 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:32.209222 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:32.209252 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:32.211170 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:32.211762 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:32.211769 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:32.211773 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:32.211876 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:32.212233 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:32.212237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:32.212239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:32.212490 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:32.212617 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:32.213062 (+ 445us) server_negotiation.cc:299] Negotiation successful
0807 15:14:32.213197 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 15:14:32.213454 16399 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b97ba2c0f46d42c29a1743c8108bd016" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 15:14:32.217069 16423 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b97ba2c0f46d42c29a1743c8108bd016: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:14:32.217357 16423 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:14:32.217722 16406 sys_catalog.cc:455] T 00000000000000000000000000000000 P b97ba2c0f46d42c29a1743c8108bd016 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6532903ceed64d74b0b7352565973d75. Latest consensus state: current_term: 1 leader_uuid: "6532903ceed64d74b0b7352565973d75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532903ceed64d74b0b7352565973d75" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46147 } } peers { permanent_uuid: "98ec48b7d46e498ba401addacc9082c9" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 } } peers { permanent_uuid: "b97ba2c0f46d42c29a1743c8108bd016" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } } }
I20260807 15:14:32.217846 16406 sys_catalog.cc:458] T 00000000000000000000000000000000 P b97ba2c0f46d42c29a1743c8108bd016 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:14:32.219010 16195 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 98ec48b7d46e498ba401addacc9082c9 [term 1 FOLLOWER]: Refusing update from remote peer 6532903ceed64d74b0b7352565973d75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:32.219617 16399 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98ec48b7d46e498ba401addacc9082c9" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:32.224414 16372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98ec48b7d46e498ba401addacc9082c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6532903ceed64d74b0b7352565973d75. Latest consensus state: current_term: 1 leader_uuid: "6532903ceed64d74b0b7352565973d75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532903ceed64d74b0b7352565973d75" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46147 } } peers { permanent_uuid: "98ec48b7d46e498ba401addacc9082c9" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 } } peers { permanent_uuid: "b97ba2c0f46d42c29a1743c8108bd016" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } } }
I20260807 15:14:32.224570 16372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98ec48b7d46e498ba401addacc9082c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:14:32.229148 16399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6532903ceed64d74b0b7352565973d75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532903ceed64d74b0b7352565973d75" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46147 } } peers { permanent_uuid: "98ec48b7d46e498ba401addacc9082c9" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 } } peers { permanent_uuid: "b97ba2c0f46d42c29a1743c8108bd016" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } } }
I20260807 15:14:32.229331 16399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:32.230715 16372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98ec48b7d46e498ba401addacc9082c9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6532903ceed64d74b0b7352565973d75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532903ceed64d74b0b7352565973d75" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46147 } } peers { permanent_uuid: "98ec48b7d46e498ba401addacc9082c9" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 } } peers { permanent_uuid: "b97ba2c0f46d42c29a1743c8108bd016" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } } }
I20260807 15:14:32.230842 16372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98ec48b7d46e498ba401addacc9082c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:14:32.231634 16351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6532903ceed64d74b0b7352565973d75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532903ceed64d74b0b7352565973d75" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46147 } } peers { permanent_uuid: "98ec48b7d46e498ba401addacc9082c9" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 } } peers { permanent_uuid: "b97ba2c0f46d42c29a1743c8108bd016" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } } }
I20260807 15:14:32.231784 16351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75 [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:32.232892 16406 sys_catalog.cc:455] T 00000000000000000000000000000000 P b97ba2c0f46d42c29a1743c8108bd016 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6532903ceed64d74b0b7352565973d75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6532903ceed64d74b0b7352565973d75" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 46147 } } peers { permanent_uuid: "98ec48b7d46e498ba401addacc9082c9" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 43999 } } peers { permanent_uuid: "b97ba2c0f46d42c29a1743c8108bd016" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34659 } } }
I20260807 15:14:32.233017 16406 sys_catalog.cc:458] T 00000000000000000000000000000000 P b97ba2c0f46d42c29a1743c8108bd016 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:14:32.233518 16405 mvcc.cc:205] Tried to move back new op lower bound from 7315929793352683520 to 7315929793250467840. Current Snapshot: MvccSnapshot[applied={T|T < 7315929793352683520}]
I20260807 15:14:32.233695 16426 mvcc.cc:205] Tried to move back new op lower bound from 7315929793352683520 to 7315929793250467840. Current Snapshot: MvccSnapshot[applied={T|T < 7315929793352683520}]
I20260807 15:14:32.234300 16428 mvcc.cc:205] Tried to move back new op lower bound from 7315929793352683520 to 7315929793250467840. Current Snapshot: MvccSnapshot[applied={T|T < 7315929793352683520}]
I20260807 15:14:32.239532 16402 catalog_manager.cc:1347] Generated new cluster ID: 0fc393cd39884907902fad5bca22c4dc
I20260807 15:14:32.239619 16402 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:32.275911 16402 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 15:14:32.277334 16402 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:32.295354 16402 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6532903ceed64d74b0b7352565973d75: Generated new TSK 0
I20260807 15:14:32.296343 16402 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 15:14:32.335156 16123 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:32.331594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46474 (local address 127.15.226.126:46147)
0807 15:14:32.331759 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:32.331763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:32.333844 (+ 2081us) server_negotiation.cc:407] Connection header received
0807 15:14:32.334097 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:32.334101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:32.334156 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:32.334214 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:32.335005 (+ 791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46474: BlockingRecv error: recv got EOF from 127.0.0.1:46474 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260807 15:14:32.336865 16234 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:32.332129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53816 (local address 127.15.226.125:43999)
0807 15:14:32.332252 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:32.332256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:32.335875 (+ 3619us) server_negotiation.cc:407] Connection header received
0807 15:14:32.336085 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:32.336088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:32.336141 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:32.336194 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:32.336736 (+ 542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53816: BlockingRecv error: recv got EOF from 127.0.0.1:53816 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 15:14:32.338713 16347 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:32.332602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35814 (local address 127.15.226.124:34659)
0807 15:14:32.332780 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:32.332785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:32.337573 (+ 4788us) server_negotiation.cc:407] Connection header received
0807 15:14:32.337881 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:32.337885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:32.337944 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:32.337998 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:32.338542 (+ 544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35814: BlockingRecv error: recv got EOF from 127.0.0.1:35814 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 15:14:32.343832 16234 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:32.342096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53830 (local address 127.15.226.125:43999)
0807 15:14:32.342210 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:32.342215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:32.342777 (+ 562us) server_negotiation.cc:407] Connection header received
0807 15:14:32.343012 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:32.343015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:32.343097 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:32.343152 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:32.343723 (+ 571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53830: BlockingRecv error: recv got EOF from 127.0.0.1:53830 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 15:14:32.349401 16234 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:32.347773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53846 (local address 127.15.226.125:43999)
0807 15:14:32.347884 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:32.347888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:32.348415 (+ 527us) server_negotiation.cc:407] Connection header received
0807 15:14:32.348625 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:32.348629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:32.348691 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:32.348756 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:32.349284 (+ 528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53846: BlockingRecv error: recv got EOF from 127.0.0.1:53846 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260807 15:14:32.355701 16234 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:32.353984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53860 (local address 127.15.226.125:43999)
0807 15:14:32.354091 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:32.354098 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:32.354683 (+ 585us) server_negotiation.cc:407] Connection header received
0807 15:14:32.354905 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:32.354909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:32.354962 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:32.355014 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:32.355588 (+ 574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53860: BlockingRecv error: recv got EOF from 127.0.0.1:53860 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260807 15:14:32.364207 16234 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:32.362250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53876 (local address 127.15.226.125:43999)
0807 15:14:32.362409 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:32.362413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:32.363113 (+ 700us) server_negotiation.cc:407] Connection header received
0807 15:14:32.363350 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:32.363353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:32.363411 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:32.363466 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:32.364065 (+ 599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53876: BlockingRecv error: recv got EOF from 127.0.0.1:53876 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260807 15:14:32.376896 16234 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:32.374965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53882 (local address 127.15.226.125:43999)
0807 15:14:32.375097 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:32.375101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:32.375766 (+ 665us) server_negotiation.cc:407] Connection header received
0807 15:14:32.375998 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:32.376003 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:32.376061 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:32.376123 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:32.376769 (+ 646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53882: BlockingRecv error: recv got EOF from 127.0.0.1:53882 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 15:14:32.398329 16234 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:32.396069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53896 (local address 127.15.226.125:43999)
0807 15:14:32.396232 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:32.396237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:32.397003 (+ 766us) server_negotiation.cc:407] Connection header received
0807 15:14:32.397326 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:32.397332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:32.397402 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:32.397467 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:32.398186 (+ 719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53896: BlockingRecv error: recv got EOF from 127.0.0.1:53896 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 15:14:32.434891 16234 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:32.433231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53902 (local address 127.15.226.125:43999)
0807 15:14:32.433376 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:32.433382 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:32.433674 (+ 292us) server_negotiation.cc:407] Connection header received
0807 15:14:32.433841 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:32.433844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:32.433900 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:32.433986 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:32.434642 (+ 656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53902: BlockingRecv error: recv got EOF from 127.0.0.1:53902 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260807 15:14:32.502554 16234 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:32.501139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53904 (local address 127.15.226.125:43999)
0807 15:14:32.501339 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:32.501344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:32.501498 (+ 154us) server_negotiation.cc:407] Connection header received
0807 15:14:32.501708 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:32.501712 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:32.501764 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:32.501863 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:32.502420 (+ 557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53904: BlockingRecv error: recv got EOF from 127.0.0.1:53904 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260807 15:14:32.634380 16234 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:32.633074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53912 (local address 127.15.226.125:43999)
0807 15:14:32.633268 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:32.633272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:32.633406 (+ 134us) server_negotiation.cc:407] Connection header received
0807 15:14:32.633545 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:32.633548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:32.633596 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:32.633689 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:32.634251 (+ 562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53912: BlockingRecv error: recv got EOF from 127.0.0.1:53912 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260807 15:14:32.893997 16234 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:32.892838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53920 (local address 127.15.226.125:43999)
0807 15:14:32.893026 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:32.893030 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:32.893095 (+ 65us) server_negotiation.cc:407] Connection header received
0807 15:14:32.893223 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:32.893225 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:32.893277 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:32.893356 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:32.893874 (+ 518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53920: BlockingRecv error: recv got EOF from 127.0.0.1:53920 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 15:14:33.173872 16391 catalog_manager.cc:1259] Loaded cluster ID: 0fc393cd39884907902fad5bca22c4dc
I20260807 15:14:33.173978 16391 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 98ec48b7d46e498ba401addacc9082c9: loading cluster ID for follower catalog manager: success
I20260807 15:14:33.175858 16391 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 98ec48b7d46e498ba401addacc9082c9: acquiring CA information for follower catalog manager: success
I20260807 15:14:33.177155 16391 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 98ec48b7d46e498ba401addacc9082c9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:14:33.219026 16423 catalog_manager.cc:1259] Loaded cluster ID: 0fc393cd39884907902fad5bca22c4dc
I20260807 15:14:33.219142 16423 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b97ba2c0f46d42c29a1743c8108bd016: loading cluster ID for follower catalog manager: success
I20260807 15:14:33.220940 16423 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b97ba2c0f46d42c29a1743c8108bd016: acquiring CA information for follower catalog manager: success
I20260807 15:14:33.222189 16423 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b97ba2c0f46d42c29a1743c8108bd016: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:14:33.412849 16517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:33.408638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53924 (local address 127.15.226.125:43999)
0807 15:14:33.408955 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:33.408959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:33.408977 (+ 18us) server_negotiation.cc:407] Connection header received
0807 15:14:33.409175 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:33.409178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:33.409326 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:33.409446 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:33.409741 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:33.410293 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:33.411249 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:33.411492 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:33.411583 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:33.411934 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:33.412398 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:33.412496 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:33.412637 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 15:14:33.412699 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":105,"threads_started":1}
I20260807 15:14:33.420872 16529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:33.416488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35824 (local address 127.15.226.124:34659)
0807 15:14:33.416792 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:33.416796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:33.416813 (+ 17us) server_negotiation.cc:407] Connection header received
0807 15:14:33.416907 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:33.416909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:33.417045 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:33.417178 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:33.417447 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:33.418032 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:33.419015 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:33.419279 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:33.419374 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:33.419801 (+ 427us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:33.420294 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:33.420402 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:33.420512 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 15:14:33.420575 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":99,"threads_started":1}
I20260807 15:14:33.429553 16517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:33.424634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53926 (local address 127.15.226.125:43999)
0807 15:14:33.424770 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:33.424775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:33.425078 (+ 303us) server_negotiation.cc:407] Connection header received
0807 15:14:33.425234 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:33.425238 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:33.425303 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:33.425393 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:33.425688 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:33.426556 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:33.428230 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:33.428556 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:33.428678 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:33.428952 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:33.429069 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:33.429147 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:33.429312 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 15:14:33.429386 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 15:14:33.429545 16535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:33.424379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46490 (local address 127.15.226.126:46147)
0807 15:14:33.424787 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:33.424792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:33.425167 (+ 375us) server_negotiation.cc:407] Connection header received
0807 15:14:33.425292 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:33.425296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:33.425486 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:33.425645 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:33.425969 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:33.426770 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:33.427717 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:33.427970 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:33.428088 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:33.428528 (+ 440us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:33.428958 (+ 430us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:33.429060 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:33.429265 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 15:14:33.429342 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":90,"threads_started":1}
I20260807 15:14:33.429771 16529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:33.424549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35840 (local address 127.15.226.124:34659)
0807 15:14:33.424780 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:33.424785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:33.425135 (+ 350us) server_negotiation.cc:407] Connection header received
0807 15:14:33.425328 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:33.425331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:33.425404 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:33.425476 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:33.425842 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:33.426483 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:33.428217 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:33.428558 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:33.428643 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:33.429086 (+ 443us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:33.429177 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:33.429243 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:33.429559 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 15:14:33.429634 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 15:14:33.433410 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 16030
I20260807 15:14:33.448923 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 16128
I20260807 15:14:33.464602 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 16239
2026-08-07T15:14:33Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3412 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 15:14:33 dist-test-slave-3kk6 krb5kdc[16543](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 15:14:33 dist-test-slave-3kk6 krb5kdc[16543](info): set up 2 sockets
Aug 07 15:14:33 dist-test-slave-3kk6 krb5kdc[16543](info): commencing operation
krb5kdc: starting...
W20260807 15:14:34.728948 16265 mini_kdc.cc:121] Time spent starting KDC: real 1.235s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/krb5kdc/test-user.keytab.
Aug 07 15:14:34 dist-test-slave-3kk6 krb5kdc[16543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T15:14:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T15:14:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.226.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
Entry for principal kudu/127.15.226.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab.
I20260807 15:14:34.871315 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:45237
--webserver_interface=127.15.226.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:39293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:45237,127.15.226.125:45833,127.15.226.124:34079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 15:14:35.002440 16560 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:14:35.002930 16560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:35.003013 16560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:35.003098 16560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:35.007351 16560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:35.007488 16560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:35.007539 16560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:35.007583 16560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:35.007617 16560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:35.013640 16560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:39293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:45237,127.15.226.125:45833,127.15.226.124:34079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.126.keytab
--principal=kudu/127.15.226.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.126:45237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.226.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:35.015012 16560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:35.016161 16560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:35.025421 16568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:35.025421 16571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:35.025473 16567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:35.025839 16560 server_base.cc:1034] running on GCE node
I20260807 15:14:35.026605 16560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:35.027958 16560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:35.029134 16560 hybrid_clock.cc:648] HybridClock initialized: now 1786115675029115 us; error 41 us; skew 500 ppm
Aug 07 15:14:35 dist-test-slave-3kk6 krb5kdc[16543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115675, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:35.032405 16560 init.cc:377] Logged in from keytab as kudu/127.15.226.126@KRBTEST.COM (short username kudu)
I20260807 15:14:35.033437 16560 webserver.cc:460] Webserver started at http://127.15.226.126:41841/ using document root <none> and password file <none>
I20260807 15:14:35.034013 16560 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:35.034078 16560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:35.034264 16560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:35.036631 16560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-0/data/instance:
uuid: "50357226b66049508d3a2f80322616dc"
format_stamp: "Formatted at 2026-08-07 15:14:35 on dist-test-slave-3kk6"
server_key: "c10e4fa1e21a618157a6b37a2cfeb340"
server_key_iv: "c7aa1ac14c0b69109e573bd080fd7ccf"
server_key_version: "encryptionkey@0"
I20260807 15:14:35.037266 16560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-0/wal/instance:
uuid: "50357226b66049508d3a2f80322616dc"
format_stamp: "Formatted at 2026-08-07 15:14:35 on dist-test-slave-3kk6"
server_key: "c10e4fa1e21a618157a6b37a2cfeb340"
server_key_iv: "c7aa1ac14c0b69109e573bd080fd7ccf"
server_key_version: "encryptionkey@0"
I20260807 15:14:35.041692 16560 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260807 15:14:35.044545 16580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:35.046080 16560 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 15:14:35.046329 16560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-0/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-0/wal
uuid: "50357226b66049508d3a2f80322616dc"
format_stamp: "Formatted at 2026-08-07 15:14:35 on dist-test-slave-3kk6"
server_key: "c10e4fa1e21a618157a6b37a2cfeb340"
server_key_iv: "c7aa1ac14c0b69109e573bd080fd7ccf"
server_key_version: "encryptionkey@0"
I20260807 15:14:35.046473 16560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:35.064697 16560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:35.068306 16560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:35.068538 16560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:35.076510 16560 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.126:45237
I20260807 15:14:35.076900 16653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.126:45237 every 8 connection(s)
I20260807 15:14:35.077668 16560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-0/data/info.pb
I20260807 15:14:35.080767 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 16560
I20260807 15:14:35.080925 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-0/wal/instance
I20260807 15:14:35.081002 16655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:35.081281 16265 external_mini_cluster.cc:1428] Setting key eb24658bc8304bab7d8c995006d4996a
I20260807 15:14:35.085973 16655 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45237 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.226.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.125@KRBTEST.COM" created.
Aug 07 15:14:35 dist-test-slave-3kk6 krb5kdc[16543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115675, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:14:35.104954 16660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.088656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47210 (local address 127.15.226.126:45237)
0807 15:14:35.089320 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.089337 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.089397 (+ 60us) server_negotiation.cc:407] Connection header received
0807 15:14:35.090594 (+ 1197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.090603 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.090946 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.091171 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.092195 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.093159 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.094629 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.095213 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.095351 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:35.098750 (+ 3399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:35.098761 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:35.098782 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:35.098819 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:35.101422 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.102060 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.102068 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.102070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.102265 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.102638 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.102645 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.102676 (+ 31us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.103317 (+ 641us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:35.103475 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:35.103905 (+ 430us) server_negotiation.cc:299] Negotiation successful
0807 15:14:35.104248 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":87,"server-negotiator.queue_time_us":528,"thread_start_us":200,"threads_started":1}
I20260807 15:14:35.104954 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.088655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:45237 (local address 127.0.0.1:47210)
0807 15:14:35.089215 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.089275 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:35.090225 (+ 950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:35.091206 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:35.091212 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:35.091610 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:35.092018 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:35.092039 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.093347 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.093351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:35.094455 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:35.094491 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.095411 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.095419 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:35.095579 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:35.096546 (+ 967us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:35.096577 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:35.098496 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:35.101607 (+ 3111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:35.101617 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:35.101627 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:35.101917 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:35.102401 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:35.102404 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:35.102405 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:35.102530 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:35.103470 (+ 940us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:35.103477 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:35.103617 (+ 140us) client_negotiation.cc:769] Sending connection context
0807 15:14:35.103772 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 15:14:35.104089 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":154,"threads_started":1}
I20260807 15:14:35.107792 16655 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:14:35.108668 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.108393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:45833 (local address 127.0.0.1:35710)
0807 15:14:35.108522 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.108601 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.125:45833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 15:14:35.108969 16581 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.226.125:45833: connect: Connection refused (error 111)
I20260807 15:14:35.109952 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.109737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:45833 (local address 127.0.0.1:35716)
0807 15:14:35.109838 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.109895 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.125:45833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 15:14:35.110114 16655 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.125:45833: Network error: Client connection negotiation failed: client connection to 127.15.226.125:45833: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.125.keytab.
Entry for principal kudu/127.15.226.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.125.keytab.
I20260807 15:14:35.114001 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.125:45833
--webserver_interface=127.15.226.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.125.keytab
--principal=kudu/127.15.226.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:39293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:45237,127.15.226.125:45833,127.15.226.124:34079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:14:35.159467 16655 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } attempt: 1
I20260807 15:14:35.160820 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.160326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:45833 (local address 127.0.0.1:35728)
0807 15:14:35.160589 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.160693 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.125:45833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260807 15:14:35.161855 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.161574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:45833 (local address 127.0.0.1:35732)
0807 15:14:35.161707 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.161773 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.125:45833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 15:14:35.162079 16655 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.125:45833: Network error: Client connection negotiation failed: client connection to 127.15.226.125:45833: connect: Connection refused (error 111)
I20260807 15:14:35.230319 16655 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } attempt: 2
I20260807 15:14:35.231464 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.231044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:45833 (local address 127.0.0.1:35744)
0807 15:14:35.231307 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.231381 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.125:45833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260807 15:14:35.232306 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.232065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:45833 (local address 127.0.0.1:35750)
0807 15:14:35.232181 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.232245 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.125:45833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 15:14:35.232448 16655 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.125:45833: Network error: Client connection negotiation failed: client connection to 127.15.226.125:45833: connect: Connection refused (error 111)
W20260807 15:14:35.240398 16666 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:14:35.240818 16666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:35.240893 16666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:35.240978 16666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:35.245402 16666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:35.245543 16666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:35.245585 16666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:35.245623 16666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:35.245659 16666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:35.252072 16666 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:39293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:45237,127.15.226.125:45833,127.15.226.124:34079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.125.keytab
--principal=kudu/127.15.226.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.125:45833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.226.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:35.253345 16666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:35.254472 16666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:35.263698 16672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:35.263847 16666 server_base.cc:1034] running on GCE node
W20260807 15:14:35.263698 16675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:35.263698 16677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:35.264709 16666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:35.266005 16666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:35.267215 16666 hybrid_clock.cc:648] HybridClock initialized: now 1786115675267190 us; error 44 us; skew 500 ppm
Aug 07 15:14:35 dist-test-slave-3kk6 krb5kdc[16543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115675, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:35.270495 16666 init.cc:377] Logged in from keytab as kudu/127.15.226.125@KRBTEST.COM (short username kudu)
I20260807 15:14:35.271592 16666 webserver.cc:460] Webserver started at http://127.15.226.125:43597/ using document root <none> and password file <none>
I20260807 15:14:35.272182 16666 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:35.272258 16666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:35.272449 16666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:35.274782 16666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-1/data/instance:
uuid: "b5b73d461b974d3cb2eeed537dabc694"
format_stamp: "Formatted at 2026-08-07 15:14:35 on dist-test-slave-3kk6"
server_key: "88d4864d1ccc44e30ab2f62427ace102"
server_key_iv: "813e286fe8b82c3eebf4daeb284d70df"
server_key_version: "encryptionkey@0"
I20260807 15:14:35.275394 16666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-1/wal/instance:
uuid: "b5b73d461b974d3cb2eeed537dabc694"
format_stamp: "Formatted at 2026-08-07 15:14:35 on dist-test-slave-3kk6"
server_key: "88d4864d1ccc44e30ab2f62427ace102"
server_key_iv: "813e286fe8b82c3eebf4daeb284d70df"
server_key_version: "encryptionkey@0"
I20260807 15:14:35.279769 16666 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 15:14:35.282619 16688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:35.284397 16666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 15:14:35.284600 16666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-1/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-1/wal
uuid: "b5b73d461b974d3cb2eeed537dabc694"
format_stamp: "Formatted at 2026-08-07 15:14:35 on dist-test-slave-3kk6"
server_key: "88d4864d1ccc44e30ab2f62427ace102"
server_key_iv: "813e286fe8b82c3eebf4daeb284d70df"
server_key_version: "encryptionkey@0"
I20260807 15:14:35.284775 16666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:35.318279 16666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:35.322585 16666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:35.322914 16666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:35.323644 16655 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } attempt: 3
I20260807 15:14:35.324723 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.324305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:45833 (local address 127.0.0.1:35758)
0807 15:14:35.324510 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.324606 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.125:45833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 15:14:35.325821 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.325485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:45833 (local address 127.0.0.1:35770)
0807 15:14:35.325643 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.325721 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.125:45833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 15:14:35.326041 16655 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.125:45833: Network error: Client connection negotiation failed: client connection to 127.15.226.125:45833: connect: Connection refused (error 111)
I20260807 15:14:35.332904 16666 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.125:45833
I20260807 15:14:35.333305 16762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.125:45833 every 8 connection(s)
I20260807 15:14:35.334227 16666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-1/data/info.pb
I20260807 15:14:35.337522 16764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:35.341984 16764 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45237 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:14:35.343612 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 16666
I20260807 15:14:35.343864 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-1/wal/instance
I20260807 15:14:35.344244 16265 external_mini_cluster.cc:1428] Setting key a2feac6736e66ec92098dc0e0d86cb28
Aug 07 15:14:35 dist-test-slave-3kk6 krb5kdc[16543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115675, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.125@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.226.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.226.124@KRBTEST.COM" created.
I20260807 15:14:35.359704 16660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.344437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47222 (local address 127.15.226.126:45237)
0807 15:14:35.344643 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.344648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.345642 (+ 994us) server_negotiation.cc:407] Connection header received
0807 15:14:35.346752 (+ 1110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.346758 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.346837 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.346963 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.348396 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.349328 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.350946 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.351334 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.351458 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:35.354948 (+ 3490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:35.354951 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:35.354953 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:35.355004 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:35.356729 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.357607 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.357610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.357611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.357815 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.358291 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.358297 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.358299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.358538 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:35.358667 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:35.359345 (+ 678us) server_negotiation.cc:299] Negotiation successful
0807 15:14:35.359517 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 15:14:35.360654 16768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.344837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:45237 (local address 127.0.0.1:47222)
0807 15:14:35.345418 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.345481 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:35.346472 (+ 991us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:35.347181 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:35.347190 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:35.347627 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:35.348176 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:35.348191 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.349519 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.349522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:35.350760 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:35.350778 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.351566 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.351573 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:35.351871 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:35.352658 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:35.352685 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:35.354612 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:35.356885 (+ 2273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:35.356892 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:35.356907 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:35.357387 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:35.357967 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:35.357972 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:35.357974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:35.358176 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:35.358714 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:35.358728 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:35.359037 (+ 309us) client_negotiation.cc:769] Sending connection context
0807 15:14:35.359285 (+ 248us) client_negotiation.cc:240] Negotiation successful
0807 15:14:35.359679 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":179,"threads_started":1}
I20260807 15:14:35.362583 16764 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:14:35 dist-test-slave-3kk6 krb5kdc[16543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115675, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.125@KRBTEST.COM for kudu/127.15.226.125@KRBTEST.COM
I20260807 15:14:35.376597 16768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.363344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:45833 (local address 127.0.0.1:35782)
0807 15:14:35.363507 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.363526 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:35.363624 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:35.364307 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:35.364310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:35.364536 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:35.364622 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:35.364628 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.365670 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.365676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:35.366236 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:35.366245 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.366967 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.366971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:35.367057 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:35.367675 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:35.367687 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:35.371861 (+ 4174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:35.374730 (+ 2869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:35.374734 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:35.374735 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:35.374968 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:35.375367 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:35.375370 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:35.375371 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:35.375474 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:35.376081 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:35.376084 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:35.376175 (+ 91us) client_negotiation.cc:769] Sending connection context
0807 15:14:35.376268 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 15:14:35.376406 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 15:14:35.376897 16772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.363473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35782 (local address 127.15.226.125:45833)
0807 15:14:35.363935 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.363944 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.363964 (+ 20us) server_negotiation.cc:407] Connection header received
0807 15:14:35.364003 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.364007 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.364176 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.364369 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.364748 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.365538 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.366370 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.366841 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.366961 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:35.372062 (+ 5101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:35.372079 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:35.372096 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:35.372125 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:35.374447 (+ 2322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.375105 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.375112 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.375113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.375207 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.375567 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.375570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.375571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.375973 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:35.376093 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:35.376507 (+ 414us) server_negotiation.cc:299] Negotiation successful
0807 15:14:35.376677 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":232,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.226.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.124.keytab.
Entry for principal kudu/127.15.226.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.124.keytab.
I20260807 15:14:35.379033 16265 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
/tmp/dist-test-taskwNTJgD/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.124:34079
--webserver_interface=127.15.226.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.124.keytab
--principal=kudu/127.15.226.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.226.84:39293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.226.126:45237,127.15.226.125:45833,127.15.226.124:34079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 15:14:35.379177 16764 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:14:35.379971 16768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.379678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34079 (local address 127.0.0.1:51434)
0807 15:14:35.379814 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.379903 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 15:14:35.380165 16689 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34079: connect: Connection refused (error 111)
I20260807 15:14:35.381172 16768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.380950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34079 (local address 127.0.0.1:51440)
0807 15:14:35.381066 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.381120 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 15:14:35.381417 16764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.124:34079: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34079: connect: Connection refused (error 111)
I20260807 15:14:35.430774 16764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } attempt: 1
I20260807 15:14:35.431753 16768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.431428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34079 (local address 127.0.0.1:51448)
0807 15:14:35.431584 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.431660 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 15:14:35.432613 16768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.432404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34079 (local address 127.0.0.1:51452)
0807 15:14:35.432512 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.432561 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 15:14:35.432793 16764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.124:34079: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34079: connect: Connection refused (error 111)
I20260807 15:14:35.469275 16655 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } attempt: 4
Aug 07 15:14:35 dist-test-slave-3kk6 krb5kdc[16543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115675, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for kudu/127.15.226.125@KRBTEST.COM
I20260807 15:14:35.482134 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.470029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:45833 (local address 127.0.0.1:35788)
0807 15:14:35.470254 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.470274 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:35.470391 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:35.470784 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:35.470791 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:35.471070 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:35.471558 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:35.471567 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.473162 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.473165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:35.473760 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:35.473769 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.474434 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.474437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:35.474524 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:35.475191 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:35.475204 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:35.476595 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:35.480002 (+ 3407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:35.480007 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:35.480009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:35.480358 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:35.480748 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:35.480752 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:35.480754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:35.480876 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:35.481534 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:35.481538 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:35.481656 (+ 118us) client_negotiation.cc:769] Sending connection context
0807 15:14:35.481773 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 15:14:35.481947 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115}
I20260807 15:14:35.482134 16772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.470154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35788 (local address 127.15.226.125:45833)
0807 15:14:35.470338 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.470343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.470366 (+ 23us) server_negotiation.cc:407] Connection header received
0807 15:14:35.470529 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.470534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.470603 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.470763 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.471746 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.473005 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.473904 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.474293 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.474429 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:35.476762 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:35.476766 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:35.476769 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:35.476801 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:35.479781 (+ 2980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.480480 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.480484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.480486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.480616 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.481151 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.481155 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.481157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.481423 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:35.481540 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:35.481848 (+ 308us) server_negotiation.cc:299] Negotiation successful
0807 15:14:35.481985 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 15:14:35.483222 16655 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:14:35.484104 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.483775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34079 (local address 127.0.0.1:51464)
0807 15:14:35.483936 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.484011 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 15:14:35.484813 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.484581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34079 (local address 127.0.0.1:51470)
0807 15:14:35.484694 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.484751 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 15:14:35.484951 16655 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.124:34079: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34079: connect: Connection refused (error 111)
I20260807 15:14:35.500952 16764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } attempt: 2
I20260807 15:14:35.501955 16768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.501605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34079 (local address 127.0.0.1:51486)
0807 15:14:35.501772 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.501872 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 15:14:35.502817 16768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.502560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34079 (local address 127.0.0.1:51502)
0807 15:14:35.502714 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.502762 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 15:14:35.503017 16764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.124:34079: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34079: connect: Connection refused (error 111)
W20260807 15:14:35.508908 16774 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 15:14:35.509271 16774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 15:14:35.509297 16774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 15:14:35.509384 16774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 15:14:35.513753 16774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 15:14:35.513850 16774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 15:14:35.513870 16774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 15:14:35.513888 16774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 15:14:35.513903 16774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 15:14:35.519960 16774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.226.84:39293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.226.126:45237,127.15.226.125:45833,127.15.226.124:34079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/krb5kdc/kudu_127.15.226.124.keytab
--principal=kudu/127.15.226.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.226.124:34079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.226.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 07 Aug 2026 13:48:26 UTC on ccycloud.quasar-cbxhxa.root.comops.site
build id 4572145
I20260807 15:14:35.521291 16774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 15:14:35.522423 16774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 15:14:35.532054 16779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:35.532059 16784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 15:14:35.532161 16780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 15:14:35.532536 16774 server_base.cc:1034] running on GCE node
I20260807 15:14:35.533247 16774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 15:14:35.535181 16774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 15:14:35.536425 16774 hybrid_clock.cc:648] HybridClock initialized: now 1786115675536386 us; error 73 us; skew 500 ppm
Aug 07 15:14:35 dist-test-slave-3kk6 krb5kdc[16543](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115675, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 15:14:35.540127 16774 init.cc:377] Logged in from keytab as kudu/127.15.226.124@KRBTEST.COM (short username kudu)
I20260807 15:14:35.541158 16774 webserver.cc:460] Webserver started at http://127.15.226.124:34115/ using document root <none> and password file <none>
I20260807 15:14:35.541733 16774 fs_manager.cc:359] Metadata directory not provided
I20260807 15:14:35.541797 16774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 15:14:35.541985 16774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 15:14:35.544130 16655 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } attempt: 1
I20260807 15:14:35.544519 16774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-2/data/instance:
uuid: "c4049cde96a8482bbcf15c9943c45d21"
format_stamp: "Formatted at 2026-08-07 15:14:35 on dist-test-slave-3kk6"
server_key: "bb2a180804b3639dda725926b8c80679"
server_key_iv: "ab7e90e78811833b1255d5a0e5268a65"
server_key_version: "encryptionkey@0"
I20260807 15:14:35.545032 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.544692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34079 (local address 127.0.0.1:51510)
0807 15:14:35.544856 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.544932 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 15:14:35.545267 16774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-2/wal/instance:
uuid: "c4049cde96a8482bbcf15c9943c45d21"
format_stamp: "Formatted at 2026-08-07 15:14:35 on dist-test-slave-3kk6"
server_key: "bb2a180804b3639dda725926b8c80679"
server_key_iv: "ab7e90e78811833b1255d5a0e5268a65"
server_key_version: "encryptionkey@0"
I20260807 15:14:35.545892 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.545626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34079 (local address 127.0.0.1:51512)
0807 15:14:35.545739 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.545791 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 15:14:35.546067 16655 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.124:34079: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34079: connect: Connection refused (error 111)
I20260807 15:14:35.550232 16774 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 15:14:35.553326 16797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 15:14:35.554888 16774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 15:14:35.555064 16774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-2/data,/tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-2/wal
uuid: "c4049cde96a8482bbcf15c9943c45d21"
format_stamp: "Formatted at 2026-08-07 15:14:35 on dist-test-slave-3kk6"
server_key: "bb2a180804b3639dda725926b8c80679"
server_key_iv: "ab7e90e78811833b1255d5a0e5268a65"
server_key_version: "encryptionkey@0"
I20260807 15:14:35.555215 16774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 15:14:35.584311 16774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 15:14:35.588301 16774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 15:14:35.588605 16774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 15:14:35.594209 16764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } attempt: 3
I20260807 15:14:35.595398 16768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.594930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34079 (local address 127.0.0.1:51526)
0807 15:14:35.595156 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.595273 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 15:14:35.596462 16768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.596047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34079 (local address 127.0.0.1:51530)
0807 15:14:35.596276 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.596360 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34079: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
W20260807 15:14:35.596715 16764 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.226.124:34079: Network error: Client connection negotiation failed: client connection to 127.15.226.124:34079: connect: Connection refused (error 111)
I20260807 15:14:35.598400 16774 rpc_server.cc:307] RPC server started. Bound to: 127.15.226.124:34079
I20260807 15:14:35.598750 16870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.226.124:34079 every 8 connection(s)
I20260807 15:14:35.599704 16774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-2/data/info.pb
I20260807 15:14:35.603015 16873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 15:14:35.607400 16873 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45237 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:14:35.608678 16265 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu as pid 16774
I20260807 15:14:35.608933 16265 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskwNTJgD/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786115365131938-16265-0/minicluster-data/master-2/wal/instance
I20260807 15:14:35.609249 16265 external_mini_cluster.cc:1428] Setting key 910032222e9949b7f058730c92e22c53
I20260807 15:14:35.613237 16655 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } attempt: 2
Aug 07 15:14:35 dist-test-slave-3kk6 krb5kdc[16543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115675, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.124@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
Aug 07 15:14:35 dist-test-slave-3kk6 krb5kdc[16543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115675, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.126@KRBTEST.COM for kudu/127.15.226.124@KRBTEST.COM
Aug 07 15:14:35 dist-test-slave-3kk6 krb5kdc[16543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.124@KRBTEST.COM
I20260807 15:14:35.624642 16659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.613815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34079 (local address 127.0.0.1:51558)
0807 15:14:35.614047 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.614065 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:35.614146 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:35.614502 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:35.614505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:35.614773 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:35.614901 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:35.614907 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.615778 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.615781 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:35.616529 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:35.616538 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.617261 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.617269 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:35.617334 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:35.617947 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:35.617960 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:35.620112 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:35.622492 (+ 2380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:35.622496 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:35.622499 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:35.622819 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:35.623224 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:35.623227 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:35.623229 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:35.623354 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:35.624192 (+ 838us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:35.624196 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:35.624292 (+ 96us) client_negotiation.cc:769] Sending connection context
0807 15:14:35.624380 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 15:14:35.624499 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154}
I20260807 15:14:35.624642 16660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.609579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47226 (local address 127.15.226.126:45237)
0807 15:14:35.609760 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.609765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.610887 (+ 1122us) server_negotiation.cc:407] Connection header received
0807 15:14:35.611980 (+ 1093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.611983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.612036 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.612121 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.613434 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.614141 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.615679 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.616001 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.616114 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:35.619876 (+ 3762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:35.619881 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:35.619884 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:35.619914 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:35.621936 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.622819 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.622824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.622826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.622913 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.623375 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.623381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.623385 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.623674 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:35.623794 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:35.624282 (+ 488us) server_negotiation.cc:299] Negotiation successful
0807 15:14:35.624442 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 15:14:35.626237 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.610077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51546 (local address 127.15.226.124:34079)
0807 15:14:35.610751 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.610780 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.610824 (+ 44us) server_negotiation.cc:407] Connection header received
0807 15:14:35.611698 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.611727 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.612003 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.612217 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.613822 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.615258 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.616353 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.616975 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.617097 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:35.621116 (+ 4019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:35.621120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:35.621123 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:35.621171 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:35.623770 (+ 2599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.624342 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.624345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.624347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.624471 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.624825 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.624829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.624830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.625034 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:35.625241 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:35.625983 (+ 742us) server_negotiation.cc:299] Negotiation successful
0807 15:14:35.626094 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":548,"spinlock_wait_cycles":52992,"thread_start_us":238,"threads_started":1}
I20260807 15:14:35.626221 16879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.610076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.126:45237 (local address 127.0.0.1:47226)
0807 15:14:35.610708 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.610769 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:35.611700 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:35.612213 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:35.612218 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:35.612752 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:35.613203 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:35.613232 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.614273 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.614276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:35.615551 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:35.615560 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.616189 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.616193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:35.616477 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:35.617442 (+ 965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:35.617469 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:35.619670 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:35.622317 (+ 2647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:35.622327 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:35.622339 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:35.622709 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:35.623031 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:35.623037 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:35.623039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:35.623247 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:35.623794 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:35.623803 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:35.624019 (+ 216us) client_negotiation.cc:769] Sending connection context
0807 15:14:35.624690 (+ 671us) client_negotiation.cc:240] Negotiation successful
0807 15:14:35.625089 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":26,"thread_start_us":170,"threads_started":1}
I20260807 15:14:35.626221 16881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.613884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51558 (local address 127.15.226.124:34079)
0807 15:14:35.614266 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.614277 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.614298 (+ 21us) server_negotiation.cc:407] Connection header received
0807 15:14:35.614336 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.614338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.614373 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.614482 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.615027 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.615662 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.616657 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.617086 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.617235 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:35.620279 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:35.620285 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:35.620299 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:35.620321 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:35.622355 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.622941 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.622948 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.622951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.623103 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.623475 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.623478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.623480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.624054 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:35.624210 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:35.625251 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0807 15:14:35.625398 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":75,"server-negotiator.queue_time_us":301,"thread_start_us":136,"threads_started":1}
I20260807 15:14:35.627846 16873 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } has no permanent_uuid. Determining permanent_uuid...
I20260807 15:14:35.630707 16655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc: Bootstrap starting.
I20260807 15:14:35.633381 16655 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc: Neither blocks nor log segments found. Creating new log.
Aug 07 15:14:35 dist-test-slave-3kk6 krb5kdc[16543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115675, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.124@KRBTEST.COM for kudu/127.15.226.125@KRBTEST.COM
I20260807 15:14:35.634402 16655 log.cc:824] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:35.637950 16655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc: No bootstrap required, opened a new log
I20260807 15:14:35.637991 16772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.628782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35796 (local address 127.15.226.125:45833)
0807 15:14:35.628924 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.628929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.628993 (+ 64us) server_negotiation.cc:407] Connection header received
0807 15:14:35.629136 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.629139 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.629195 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.629303 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.629784 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.630319 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.631243 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.631488 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.631605 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:35.634017 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:35.634020 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:35.634022 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:35.634048 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:35.636141 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.636672 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.636676 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.636678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.636774 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.637094 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.637097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.637099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.637337 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:35.637451 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:35.637757 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 15:14:35.637869 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 15:14:35.638024 16879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.628646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.125:45833 (local address 127.0.0.1:35796)
0807 15:14:35.628929 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.628946 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:35.629039 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:35.629352 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:35.629355 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:35.629554 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:35.629658 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:35.629664 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.630437 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.630442 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:35.631120 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:35.631129 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.631609 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.631613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:35.631730 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:35.632304 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:35.632317 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:35.633860 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:35.636283 (+ 2423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:35.636287 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:35.636290 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:35.636560 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:35.636883 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:35.636886 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:35.636888 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:35.636987 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:35.637459 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:35.637462 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:35.637576 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 15:14:35.637699 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 15:14:35.637853 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213}
I20260807 15:14:35.638891 16873 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 15:14:35 dist-test-slave-3kk6 krb5kdc[16543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.126@KRBTEST.COM
I20260807 15:14:35.643234 16655 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50357226b66049508d3a2f80322616dc" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45237 } } peers { permanent_uuid: "b5b73d461b974d3cb2eeed537dabc694" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } } peers { permanent_uuid: "c4049cde96a8482bbcf15c9943c45d21" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } }
I20260807 15:14:35.643607 16655 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50357226b66049508d3a2f80322616dc, State: Initialized, Role: FOLLOWER
I20260807 15:14:35.644074 16655 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50357226b66049508d3a2f80322616dc" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45237 } } peers { permanent_uuid: "b5b73d461b974d3cb2eeed537dabc694" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } } peers { permanent_uuid: "c4049cde96a8482bbcf15c9943c45d21" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } }
Aug 07 15:14:35 dist-test-slave-3kk6 krb5kdc[16543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115675, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.124@KRBTEST.COM for kudu/127.15.226.124@KRBTEST.COM
I20260807 15:14:35.645000 16883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50357226b66049508d3a2f80322616dc" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45237 } } peers { permanent_uuid: "b5b73d461b974d3cb2eeed537dabc694" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } } peers { permanent_uuid: "c4049cde96a8482bbcf15c9943c45d21" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } } }
I20260807 15:14:35.645601 16883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:14:35.646270 16655 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:35.646615 16660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.634657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47234 (local address 127.15.226.126:45237)
0807 15:14:35.634815 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.634820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.634945 (+ 125us) server_negotiation.cc:407] Connection header received
0807 15:14:35.635061 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.635067 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.635132 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.635228 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.635903 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.636408 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.638346 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.638691 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.638800 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:35.642296 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:35.642300 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:35.642303 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:35.642343 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:35.644118 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.644610 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.644614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.644616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.644709 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.645091 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.645095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.645097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.645362 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:35.645563 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:35.646366 (+ 803us) server_negotiation.cc:299] Negotiation successful
0807 15:14:35.646498 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 15:14:35.649067 16879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.639466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34079 (local address 127.0.0.1:51562)
0807 15:14:35.639583 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.639598 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:35.639677 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:35.639889 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:35.639892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:35.640073 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:35.640140 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:35.640145 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.640892 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.640897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:35.641761 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:35.641774 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.642522 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.642525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:35.642595 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:35.643148 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:35.643164 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:35.644710 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:35.646818 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:35.646823 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:35.646825 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:35.647102 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:35.647428 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:35.647430 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:35.647432 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:35.647850 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:35.648449 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:35.648451 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:35.648543 (+ 92us) client_negotiation.cc:769] Sending connection context
0807 15:14:35.648657 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 15:14:35.648837 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 15:14:35.649714 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.639595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51562 (local address 127.15.226.124:34079)
0807 15:14:35.639690 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.639696 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.639716 (+ 20us) server_negotiation.cc:407] Connection header received
0807 15:14:35.639746 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.639748 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.639782 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.639879 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.640254 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.640766 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.641906 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.642293 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.642402 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:35.644865 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:35.644868 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:35.644871 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:35.644896 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:35.646635 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.647236 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.647239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.647241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.647328 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.647975 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.647978 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.647980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.648310 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:35.648413 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:35.649386 (+ 973us) server_negotiation.cc:299] Negotiation successful
0807 15:14:35.649542 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 15:14:35.653695 16873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4049cde96a8482bbcf15c9943c45d21: Bootstrap starting.
W20260807 15:14:35.654451 16897 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:14:35.654518 16897 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:14:35.654403 16560 master_runner.cc:186] Error getting master registration for 127.15.226.125:45833: OK, instance_id { permanent_uuid: "b5b73d461b974d3cb2eeed537dabc694" instance_seqno: 1786115675329958 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:14:35.656982 16873 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4049cde96a8482bbcf15c9943c45d21: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:35.657058 16560 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:14:35.658041 16873 log.cc:824] T 00000000000000000000000000000000 P c4049cde96a8482bbcf15c9943c45d21: Log is configured to *not* fsync() on all Append() calls
Aug 07 15:14:35 dist-test-slave-3kk6 krb5kdc[16543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.226.125@KRBTEST.COM
I20260807 15:14:35.661710 16873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4049cde96a8482bbcf15c9943c45d21: No bootstrap required, opened a new log
I20260807 15:14:35.664392 16772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.648165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35808 (local address 127.15.226.125:45833)
0807 15:14:35.648283 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.648288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.648342 (+ 54us) server_negotiation.cc:407] Connection header received
0807 15:14:35.648484 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.648487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.648544 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.648636 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.649691 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.650420 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.652326 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.652697 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.652786 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:35.659765 (+ 6979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:35.659769 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:35.659772 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:35.659809 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:35.661957 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.662693 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.662698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.662700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.662814 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.663185 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.663189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.663190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.663450 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:35.663731 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:35.663885 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 15:14:35.664023 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 15:14:35.666571 16873 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4049cde96a8482bbcf15c9943c45d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50357226b66049508d3a2f80322616dc" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45237 } } peers { permanent_uuid: "b5b73d461b974d3cb2eeed537dabc694" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } } peers { permanent_uuid: "c4049cde96a8482bbcf15c9943c45d21" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } }
I20260807 15:14:35.666987 16873 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4049cde96a8482bbcf15c9943c45d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4049cde96a8482bbcf15c9943c45d21, State: Initialized, Role: FOLLOWER
I20260807 15:14:35.667479 16873 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4049cde96a8482bbcf15c9943c45d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50357226b66049508d3a2f80322616dc" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45237 } } peers { permanent_uuid: "b5b73d461b974d3cb2eeed537dabc694" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } } peers { permanent_uuid: "c4049cde96a8482bbcf15c9943c45d21" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } }
I20260807 15:14:35.668097 16905 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4049cde96a8482bbcf15c9943c45d21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50357226b66049508d3a2f80322616dc" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45237 } } peers { permanent_uuid: "b5b73d461b974d3cb2eeed537dabc694" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } } peers { permanent_uuid: "c4049cde96a8482bbcf15c9943c45d21" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } } }
I20260807 15:14:35.668303 16905 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4049cde96a8482bbcf15c9943c45d21 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:14:35.668740 16265 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 15:14:35.669216 16873 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4049cde96a8482bbcf15c9943c45d21 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:35.670101 16615 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "50357226b66049508d3a2f80322616dc"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47234
I20260807 15:14:35.670406 16615 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 15:14:35.670581 16615 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:35.672467 16615 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50357226b66049508d3a2f80322616dc" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45237 } } peers { permanent_uuid: "b5b73d461b974d3cb2eeed537dabc694" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } } peers { permanent_uuid: "c4049cde96a8482bbcf15c9943c45d21" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } }
I20260807 15:14:35.673558 16615 leader_election.cc:288] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc [CANDIDATE]: Term 1 election: Requested vote from peers b5b73d461b974d3cb2eeed537dabc694 (127.15.226.125:45833), c4049cde96a8482bbcf15c9943c45d21 (127.15.226.124:34079)
I20260807 15:14:35.674597 16839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "50357226b66049508d3a2f80322616dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c4049cde96a8482bbcf15c9943c45d21"
I20260807 15:14:35.674921 16839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4049cde96a8482bbcf15c9943c45d21 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:35.676579 16839 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c4049cde96a8482bbcf15c9943c45d21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50357226b66049508d3a2f80322616dc in term 1.
I20260807 15:14:35.679003 16581 leader_election.cc:302] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50357226b66049508d3a2f80322616dc, c4049cde96a8482bbcf15c9943c45d21; no voters:
I20260807 15:14:35.679350 16883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 15:14:35.676424 16774 master_runner.cc:186] Error getting master registration for 127.15.226.125:45833: OK, instance_id { permanent_uuid: "b5b73d461b974d3cb2eeed537dabc694" instance_seqno: 1786115675329958 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 15:14:35.679594 16883 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc [term 1 LEADER]: Becoming Leader. State: Replica: 50357226b66049508d3a2f80322616dc, State: Running, Role: LEADER
I20260807 15:14:35.679687 16774 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 15:14:35.679980 16883 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50357226b66049508d3a2f80322616dc" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45237 } } peers { permanent_uuid: "b5b73d461b974d3cb2eeed537dabc694" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } } peers { permanent_uuid: "c4049cde96a8482bbcf15c9943c45d21" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } }
W20260807 15:14:35.680868 16923 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c4049cde96a8482bbcf15c9943c45d21: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:14:35.680938 16923 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:14:35.681849 16883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50357226b66049508d3a2f80322616dc. Latest consensus state: current_term: 1 leader_uuid: "50357226b66049508d3a2f80322616dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50357226b66049508d3a2f80322616dc" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45237 } } peers { permanent_uuid: "b5b73d461b974d3cb2eeed537dabc694" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } } peers { permanent_uuid: "c4049cde96a8482bbcf15c9943c45d21" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } } }
I20260807 15:14:35.681962 16883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:35.683890 16935 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 15:14:35.685094 16935 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 15:14:35.687477 16660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.676190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47242 (local address 127.15.226.126:45237)
0807 15:14:35.676321 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.676327 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.677108 (+ 781us) server_negotiation.cc:407] Connection header received
0807 15:14:35.677301 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.677304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.677367 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.677462 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.678381 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.679079 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.680947 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.681310 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.681387 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:35.683527 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:35.683531 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:35.683534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:35.683564 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:35.685402 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.685946 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.685950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.685951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.686045 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.686392 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.686396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.686398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.686638 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:35.686774 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:35.687161 (+ 387us) server_negotiation.cc:299] Negotiation successful
0807 15:14:35.687314 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 15:14:35.699442 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.684163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51570 (local address 127.15.226.124:34079)
0807 15:14:35.684277 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.684281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.689182 (+ 4901us) server_negotiation.cc:407] Connection header received
0807 15:14:35.689439 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.689443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.689496 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.689549 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.690527 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.691317 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.692479 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.692826 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.692906 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:35.694945 (+ 2039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:35.694950 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:35.694953 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:35.694987 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:35.696736 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.697868 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.697872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.697874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.697973 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.698293 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.698296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.698298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.698527 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:35.698636 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:35.699092 (+ 456us) server_negotiation.cc:299] Negotiation successful
0807 15:14:35.699256 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 15:14:35.702318 16839 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c4049cde96a8482bbcf15c9943c45d21 [term 1 FOLLOWER]: Refusing update from remote peer 50357226b66049508d3a2f80322616dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:35.702780 16772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.683212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35824 (local address 127.15.226.125:45833)
0807 15:14:35.683335 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.683340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.688080 (+ 4740us) server_negotiation.cc:407] Connection header received
0807 15:14:35.688303 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.688306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.688372 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.688436 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.689673 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.690514 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.693156 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.693520 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.693613 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:35.697533 (+ 3920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:35.697537 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:35.697541 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:35.697576 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:35.699548 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.701152 (+ 1604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.701156 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.701158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.701281 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.701641 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.701647 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.701651 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.701910 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:35.702052 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:35.702433 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 15:14:35.702582 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 15:14:35.703971 16883 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4049cde96a8482bbcf15c9943c45d21" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:35.709113 16905 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4049cde96a8482bbcf15c9943c45d21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50357226b66049508d3a2f80322616dc. Latest consensus state: current_term: 1 leader_uuid: "50357226b66049508d3a2f80322616dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50357226b66049508d3a2f80322616dc" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45237 } } peers { permanent_uuid: "b5b73d461b974d3cb2eeed537dabc694" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } } peers { permanent_uuid: "c4049cde96a8482bbcf15c9943c45d21" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } } }
I20260807 15:14:35.709298 16905 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4049cde96a8482bbcf15c9943c45d21 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:14:35.713433 16932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "50357226b66049508d3a2f80322616dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50357226b66049508d3a2f80322616dc" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45237 } } peers { permanent_uuid: "b5b73d461b974d3cb2eeed537dabc694" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } } peers { permanent_uuid: "c4049cde96a8482bbcf15c9943c45d21" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } } }
I20260807 15:14:35.713606 16932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc [sys.catalog]: This master's current role is: LEADER
I20260807 15:14:35.713752 16938 mvcc.cc:205] Tried to move back new op lower bound from 7315929807651766272 to 7315929807588995072. Current Snapshot: MvccSnapshot[applied={T|T < 7315929807651766272}]
I20260807 15:14:35.713759 16905 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4049cde96a8482bbcf15c9943c45d21 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "50357226b66049508d3a2f80322616dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50357226b66049508d3a2f80322616dc" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45237 } } peers { permanent_uuid: "b5b73d461b974d3cb2eeed537dabc694" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } } peers { permanent_uuid: "c4049cde96a8482bbcf15c9943c45d21" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } } }
I20260807 15:14:35.713877 16905 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4049cde96a8482bbcf15c9943c45d21 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:14:35.716274 16939 mvcc.cc:205] Tried to move back new op lower bound from 7315929807651766272 to 7315929807588995072. Current Snapshot: MvccSnapshot[applied={T|T < 7315929807651766272}]
I20260807 15:14:35.718461 16935 catalog_manager.cc:1347] Generated new cluster ID: 065f0b1ce1a342f4a11aa3cf5a20a5b0
I20260807 15:14:35.718544 16935 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 15:14:35.739888 16764 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } attempt: 4
I20260807 15:14:35.745409 16935 catalog_manager.cc:1370] Generated new certificate authority record
Aug 07 15:14:35 dist-test-slave-3kk6 krb5kdc[16543](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786115675, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.226.125@KRBTEST.COM for kudu/127.15.226.124@KRBTEST.COM
I20260807 15:14:35.748682 16935 catalog_manager.cc:1504] Loading token signing keys...
I20260807 15:14:35.754885 16768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.740387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.226.124:34079 (local address 127.0.0.1:51572)
0807 15:14:35.740579 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 15:14:35.740596 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 15:14:35.740699 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 15:14:35.741050 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 15:14:35.741053 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 15:14:35.741306 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0807 15:14:35.741460 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:35.741467 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.742517 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.742520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:35.743291 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 15:14:35.743305 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.745223 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.745227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 15:14:35.745322 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:35.746084 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 15:14:35.746103 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 15:14:35.747859 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 15:14:35.750359 (+ 2500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:35.750363 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:35.750366 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:35.750682 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:35.753634 (+ 2952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 15:14:35.753638 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 15:14:35.753640 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 15:14:35.753761 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 15:14:35.754287 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 15:14:35.754291 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 15:14:35.754414 (+ 123us) client_negotiation.cc:769] Sending connection context
0807 15:14:35.754501 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 15:14:35.754692 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260807 15:14:35.755512 16880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:35.740473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51572 (local address 127.15.226.124:34079)
0807 15:14:35.740580 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.740586 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.740646 (+ 60us) server_negotiation.cc:407] Connection header received
0807 15:14:35.740786 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.740789 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.740839 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.740912 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.741595 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.742400 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.744751 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:35.745099 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:35.745210 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:35.748076 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 15:14:35.748082 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 15:14:35.748084 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 15:14:35.748118 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 15:14:35.750146 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.753381 (+ 3235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.753385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.753387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.753508 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 15:14:35.753885 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 15:14:35.753901 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 15:14:35.753903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 15:14:35.754137 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 15:14:35.754228 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:35.755207 (+ 979us) server_negotiation.cc:299] Negotiation successful
0807 15:14:35.755355 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 15:14:35.759049 16764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5b73d461b974d3cb2eeed537dabc694: Bootstrap starting.
I20260807 15:14:35.762384 16764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5b73d461b974d3cb2eeed537dabc694: Neither blocks nor log segments found. Creating new log.
I20260807 15:14:35.766623 16764 log.cc:824] T 00000000000000000000000000000000 P b5b73d461b974d3cb2eeed537dabc694: Log is configured to *not* fsync() on all Append() calls
I20260807 15:14:35.769773 16764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5b73d461b974d3cb2eeed537dabc694: No bootstrap required, opened a new log
I20260807 15:14:35.775338 16764 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5b73d461b974d3cb2eeed537dabc694 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50357226b66049508d3a2f80322616dc" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45237 } } peers { permanent_uuid: "b5b73d461b974d3cb2eeed537dabc694" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } } peers { permanent_uuid: "c4049cde96a8482bbcf15c9943c45d21" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } }
I20260807 15:14:35.775687 16764 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5b73d461b974d3cb2eeed537dabc694 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b73d461b974d3cb2eeed537dabc694, State: Initialized, Role: FOLLOWER
I20260807 15:14:35.776088 16764 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5b73d461b974d3cb2eeed537dabc694 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50357226b66049508d3a2f80322616dc" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45237 } } peers { permanent_uuid: "b5b73d461b974d3cb2eeed537dabc694" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } } peers { permanent_uuid: "c4049cde96a8482bbcf15c9943c45d21" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } }
I20260807 15:14:35.778334 16935 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc: Generated new TSK 0
I20260807 15:14:35.780438 16935 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 15:14:35.781333 16764 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5b73d461b974d3cb2eeed537dabc694 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 15:14:35.781379 16728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "50357226b66049508d3a2f80322616dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b5b73d461b974d3cb2eeed537dabc694"
I20260807 15:14:35.781623 16728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5b73d461b974d3cb2eeed537dabc694 [term 0 FOLLOWER]: Advancing to term 1
I20260807 15:14:35.783411 16728 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b5b73d461b974d3cb2eeed537dabc694 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50357226b66049508d3a2f80322616dc in term 1.
I20260807 15:14:35.784049 16726 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b5b73d461b974d3cb2eeed537dabc694 [term 1 FOLLOWER]: Refusing update from remote peer 50357226b66049508d3a2f80322616dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 15:14:35.786345 16883 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5b73d461b974d3cb2eeed537dabc694" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 15:14:35.789885 16947 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5b73d461b974d3cb2eeed537dabc694 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50357226b66049508d3a2f80322616dc" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45237 } } peers { permanent_uuid: "b5b73d461b974d3cb2eeed537dabc694" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } } peers { permanent_uuid: "c4049cde96a8482bbcf15c9943c45d21" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } } }
I20260807 15:14:35.790037 16947 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5b73d461b974d3cb2eeed537dabc694 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 15:14:35.790855 16962 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b5b73d461b974d3cb2eeed537dabc694: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 15:14:35.790933 16962 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 15:14:35.800262 16947 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5b73d461b974d3cb2eeed537dabc694 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50357226b66049508d3a2f80322616dc. Latest consensus state: current_term: 1 leader_uuid: "50357226b66049508d3a2f80322616dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50357226b66049508d3a2f80322616dc" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45237 } } peers { permanent_uuid: "b5b73d461b974d3cb2eeed537dabc694" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } } peers { permanent_uuid: "c4049cde96a8482bbcf15c9943c45d21" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } } }
I20260807 15:14:35.800427 16947 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5b73d461b974d3cb2eeed537dabc694 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:14:35.800545 16947 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5b73d461b974d3cb2eeed537dabc694 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "50357226b66049508d3a2f80322616dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50357226b66049508d3a2f80322616dc" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45237 } } peers { permanent_uuid: "b5b73d461b974d3cb2eeed537dabc694" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } } peers { permanent_uuid: "c4049cde96a8482bbcf15c9943c45d21" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } } }
I20260807 15:14:35.800623 16947 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5b73d461b974d3cb2eeed537dabc694 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 15:14:35.807875 16964 mvcc.cc:205] Tried to move back new op lower bound from 7315929807954432000 to 7315929807588995072. Current Snapshot: MvccSnapshot[applied={T|T < 7315929807651766272}]
W20260807 15:14:35.827405 16660 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:35.825272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47246 (local address 127.15.226.126:45237)
0807 15:14:35.825413 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.825418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.826185 (+ 767us) server_negotiation.cc:407] Connection header received
0807 15:14:35.826394 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.826398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.826451 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.826505 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.827251 (+ 746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47246: BlockingRecv error: recv got EOF from 127.0.0.1:47246 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 15:14:35.830046 16772 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:35.828165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35838 (local address 127.15.226.125:45833)
0807 15:14:35.828279 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.828283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.829013 (+ 730us) server_negotiation.cc:407] Connection header received
0807 15:14:35.829201 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.829204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.829256 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.829308 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.829900 (+ 592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35838: BlockingRecv error: recv got EOF from 127.0.0.1:35838 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260807 15:14:35.831796 16880 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:35.828711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51588 (local address 127.15.226.124:34079)
0807 15:14:35.828840 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.828845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.830754 (+ 1909us) server_negotiation.cc:407] Connection header received
0807 15:14:35.830994 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.830997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.831050 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.831103 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.831637 (+ 534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51588: BlockingRecv error: recv got EOF from 127.0.0.1:51588 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260807 15:14:35.836719 16772 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:35.834967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35852 (local address 127.15.226.125:45833)
0807 15:14:35.835091 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.835096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.835601 (+ 505us) server_negotiation.cc:407] Connection header received
0807 15:14:35.835863 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.835867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.835953 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.836010 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.836595 (+ 585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35852: BlockingRecv error: recv got EOF from 127.0.0.1:35852 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 15:14:35.842548 16772 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:35.840753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35860 (local address 127.15.226.125:45833)
0807 15:14:35.840888 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.840892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.841487 (+ 595us) server_negotiation.cc:407] Connection header received
0807 15:14:35.841694 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.841697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.841755 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.841808 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.842423 (+ 615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35860: BlockingRecv error: recv got EOF from 127.0.0.1:35860 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 15:14:35.849182 16772 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:35.847380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35870 (local address 127.15.226.125:45833)
0807 15:14:35.847505 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.847509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.848092 (+ 583us) server_negotiation.cc:407] Connection header received
0807 15:14:35.848322 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.848325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.848389 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.848446 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.849053 (+ 607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35870: BlockingRecv error: recv got EOF from 127.0.0.1:35870 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 15:14:35.857925 16772 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:35.856092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35872 (local address 127.15.226.125:45833)
0807 15:14:35.856230 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.856235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.856826 (+ 591us) server_negotiation.cc:407] Connection header received
0807 15:14:35.857063 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.857067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.857125 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.857180 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.857786 (+ 606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35872: BlockingRecv error: recv got EOF from 127.0.0.1:35872 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260807 15:14:35.870950 16772 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:35.868909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35886 (local address 127.15.226.125:45833)
0807 15:14:35.869051 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.869056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.869724 (+ 668us) server_negotiation.cc:407] Connection header received
0807 15:14:35.869977 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.869981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.870040 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.870096 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.870805 (+ 709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35886: BlockingRecv error: recv got EOF from 127.0.0.1:35886 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 15:14:35.891578 16772 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:35.890070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35902 (local address 127.15.226.125:45833)
0807 15:14:35.890211 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.890216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.890530 (+ 314us) server_negotiation.cc:407] Connection header received
0807 15:14:35.890693 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.890698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.890777 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.890875 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.891442 (+ 567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35902: BlockingRecv error: recv got EOF from 127.0.0.1:35902 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260807 15:14:35.927752 16772 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:35.926284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35908 (local address 127.15.226.125:45833)
0807 15:14:35.926466 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.926471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.926621 (+ 150us) server_negotiation.cc:407] Connection header received
0807 15:14:35.926851 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.926857 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.926942 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.927073 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.927600 (+ 527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35908: BlockingRecv error: recv got EOF from 127.0.0.1:35908 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 15:14:35.996040 16772 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:35.994494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35920 (local address 127.15.226.125:45833)
0807 15:14:35.994810 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:35.994815 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:35.995025 (+ 210us) server_negotiation.cc:407] Connection header received
0807 15:14:35.995222 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:35.995232 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:35.995284 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:35.995400 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:35.995919 (+ 519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35920: BlockingRecv error: recv got EOF from 127.0.0.1:35920 (error 108)
Metrics: {"server-negotiator.queue_time_us":205}
W20260807 15:14:36.128039 16772 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:36.126630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35926 (local address 127.15.226.125:45833)
0807 15:14:36.126867 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:36.126896 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:36.126982 (+ 86us) server_negotiation.cc:407] Connection header received
0807 15:14:36.127160 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:36.127169 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:36.127219 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:36.127314 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:36.127809 (+ 495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35926: BlockingRecv error: recv got EOF from 127.0.0.1:35926 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
I20260807 15:14:36.380491 16883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "50357226b66049508d3a2f80322616dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50357226b66049508d3a2f80322616dc" member_type: VOTER last_known_addr { host: "127.15.226.126" port: 45237 } } peers { permanent_uuid: "b5b73d461b974d3cb2eeed537dabc694" member_type: VOTER last_known_addr { host: "127.15.226.125" port: 45833 } } peers { permanent_uuid: "c4049cde96a8482bbcf15c9943c45d21" member_type: VOTER last_known_addr { host: "127.15.226.124" port: 34079 } } }
I20260807 15:14:36.380749 16883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50357226b66049508d3a2f80322616dc [sys.catalog]: This master's current role is: LEADER
W20260807 15:14:36.387761 16772 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 15:14:36.386448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35938 (local address 127.15.226.125:45833)
0807 15:14:36.386638 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:36.386642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:36.386688 (+ 46us) server_negotiation.cc:407] Connection header received
0807 15:14:36.386924 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:36.386927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:36.386981 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:36.387064 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 15:14:36.387639 (+ 575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35938: BlockingRecv error: recv got EOF from 127.0.0.1:35938 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 15:14:36.682588 16923 catalog_manager.cc:1259] Loaded cluster ID: 065f0b1ce1a342f4a11aa3cf5a20a5b0
I20260807 15:14:36.682780 16923 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c4049cde96a8482bbcf15c9943c45d21: loading cluster ID for follower catalog manager: success
I20260807 15:14:36.684635 16923 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c4049cde96a8482bbcf15c9943c45d21: acquiring CA information for follower catalog manager: success
I20260807 15:14:36.685899 16923 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c4049cde96a8482bbcf15c9943c45d21: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:14:36.792366 16962 catalog_manager.cc:1259] Loaded cluster ID: 065f0b1ce1a342f4a11aa3cf5a20a5b0
I20260807 15:14:36.792466 16962 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b5b73d461b974d3cb2eeed537dabc694: loading cluster ID for follower catalog manager: success
I20260807 15:14:36.794241 16962 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b5b73d461b974d3cb2eeed537dabc694: acquiring CA information for follower catalog manager: success
I20260807 15:14:36.795548 16962 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b5b73d461b974d3cb2eeed537dabc694: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 15:14:36.906235 17061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:36.902071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35954 (local address 127.15.226.125:45833)
0807 15:14:36.902334 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:36.902338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:36.902481 (+ 143us) server_negotiation.cc:407] Connection header received
0807 15:14:36.902607 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:36.902609 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:36.902855 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:36.902969 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:36.903230 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:36.903802 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:36.904647 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:36.904915 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:36.905000 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:36.905310 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:36.905802 (+ 492us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:36.905903 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:36.905958 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 15:14:36.906013 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":85,"threads_started":1}
I20260807 15:14:36.913583 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:36.909596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51598 (local address 127.15.226.124:34079)
0807 15:14:36.909891 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:36.909895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:36.909909 (+ 14us) server_negotiation.cc:407] Connection header received
0807 15:14:36.909993 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:36.909995 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:36.910113 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:36.910219 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:36.910421 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:36.911014 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:36.911859 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:36.912092 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:36.912179 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:36.912459 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:36.913030 (+ 571us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:36.913148 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:36.913298 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 15:14:36.913358 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":110,"threads_started":1}
I20260807 15:14:36.921375 17071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:36.917592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51604 (local address 127.15.226.124:34079)
0807 15:14:36.917720 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:36.917726 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:36.917741 (+ 15us) server_negotiation.cc:407] Connection header received
0807 15:14:36.917786 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:36.917789 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:36.917832 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:36.917898 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:36.918267 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:36.919049 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:36.920028 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:36.920363 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:36.920453 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:36.920798 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:36.920886 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:36.920958 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:36.921125 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 15:14:36.921196 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 15:14:36.921557 17061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:36.916938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35962 (local address 127.15.226.125:45833)
0807 15:14:36.917145 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:36.917150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:36.917401 (+ 251us) server_negotiation.cc:407] Connection header received
0807 15:14:36.917626 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:36.917630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:36.917697 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:36.917779 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:36.918172 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:36.919046 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:36.920240 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:36.920578 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:36.920653 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:36.921012 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:36.921124 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:36.921196 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:36.921358 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 15:14:36.921430 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260807 15:14:36.921762 17080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 15:14:36.916890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47252 (local address 127.15.226.126:45237)
0807 15:14:36.917130 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 15:14:36.917135 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 15:14:36.917236 (+ 101us) server_negotiation.cc:407] Connection header received
0807 15:14:36.917475 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 15:14:36.917481 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 15:14:36.917662 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 15:14:36.917794 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 15:14:36.918062 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:36.918727 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:36.919967 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 15:14:36.920251 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 15:14:36.920371 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 15:14:36.920742 (+ 371us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 15:14:36.921123 (+ 381us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 15:14:36.921207 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 15:14:36.921565 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 15:14:36.921644 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":76,"threads_started":1}
I20260807 15:14:36.929113 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 16560
I20260807 15:14:36.944877 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 16666
I20260807 15:14:36.958287 16265 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskwNTJgD/build/debug/bin/kudu with pid 16774
2026-08-07T15:14:36Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3493 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6906 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (311812 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST